Pilot - Interviewer Training Manual

Pilot - Appendix A interviewer training manual.docx

Evaluating Student Need for Developmental or Remedial Courses at Postsecondary Education Institutions (Formerly titled Survey of Placement Tests and Cut-Scores in Higher Education Institutions).

Pilot - Interviewer Training Manual

OMB: 3098-0006

Document [docx]

Download: docx | pdf

Appendix A: Interviewer training manual

National Assessment Governing Board (NAGB)

Evaluating Student Need for Developmental or Remedial Courses at Postsecondary Education Institutions

Telephone Interviewer Manual

Project Number: 8756.11.01

Interviewer toll-free number: 800-937-8288, plus extension

National Assessment Governing Board Contact:

Westat Contacts for NAGB:

Project Director: Dr. Basmat Parsad, 1-800-937-8281 ext. 8222, TA 2096

Survey manager: Liam Ristow, 1-800-937-8281 ext. 2456, TA 2054

Operations manager: Anjali Pandit, 301-294-4420, TA 2084

Case management supervisor: Suzanne Rodriguez, 301-251-2293, TA 1009F

August 2010

1. INTRODUCTION

1.1 Background and Purpose of the Study

This survey examines the tests and test scores used by two-year and four-year postsecondary institutions to evaluate whether entering students are academically prepared for entry-level college courses or in need of developmental or remedial courses in mathematics or reading. The survey is part of a National Assessment Governing Board (NAGB) research program, involving almost 20 studies, on the academic preparedness of 12th-grade students. The purpose of this program of research is to enable the National Assessment of Educational Progress (NAEP) to report on the academic preparedness of 12^th graders for postsecondary education and training. The Governing Board plans to use the data on tests and test scores, along with the results of the other studies, to determine how higher education institutions define student preparedness for entry-level college coursework.

Postsecondary institutions use a variety of tests to make determinations about student preparedness, including ACT and SAT, placement tests such as ACCUPLACER, ASSET, and COMPASS, and tests developed within the institution or state. We are focusing on test scores below which institutions consider a student in need of developmental or remedial courses. The working assumption in this survey is that this is also the score above which a student is deemed prepared for entry-level college courses.

The activities described in this training manual are for a pilot test of the survey instrument. The pilot test is being conducted with 120 institutions in order to explore some of the issues that arose during survey development work. A full survey administration with a nationally representative sample of institutions is planned for fall and winter 2010–11.

1.2 Sample and Mailout Activities

The pilot test sample consists of 120 two-year and four-year postsecondary institutions, including public, private not-for-profit, and private for-profit institutions. The goal in drawing the pilot test sample was to include a diverse array of institutions that would yield a range of survey responses. Table 1 displays the number of institutions in the sample and expected number of respondents by institution type.

Table 1. Proposed target sample sizes for the pilot test, by subgroup

Subgroup	Number sampled	Estimated number of respondents
Total sample	120	102
Public	60	51
4-year	30	26
2-year	30	25
Private, not-for-profit	40	34
4-year	30	25
2-year	10	9
Private, for-profit	20	17
4-year	10	9
2-year	10	8

Survey packages were mailed to the President/Chancellor’s office of the sampled postsecondary education institutions on August 2, 2010. There are two versions of the questionnaire: one version for the two-year institutions and one for the four-year institutions. The differences in the questionnaire are very slight and will be covered in Chapter 3.

1.3 Overview of Study Tasks

A cover letter to the President/Chancellor (Attachment 1), an ivory-colored questionnaire for two-year institutions, (Attachment 3), and a yellow-colored questionnaire for four-year institutions (Attachment 4) were sent to the president/chancellor’s office of each sampled institution on August 2, 2010. The cover letter requested the institution's participation and asked that the survey and accompanying materials be forwarded to the staff member or office that is responsible for the evaluation of entering students to determine need for developmental or remedial courses in mathematics and reading.

The office or staff member that handles the evaluation of entering students differs across institutions. Respondents may be from the student services office, placement office, math or English department, or admissions office. Page 3-3 in Chapter 3 gives more details on identifying the preferred survey respondent. Respondents are encouraged to complete the questionnaire online using the URL, User ID, and Password provided. They are also given the option to complete and return the questionnaire to Westat using the prepaid envelope within three weeks and to keep a copy of the completed survey for their records.

1.4 Overview of Interviewer Tasks

Obtaining data that are representative of all institutions is critical to the study and depends upon your ability to solicit the cooperation of each respondent in the sample in a limited amount of time and to follow the prescribed interviewing instructions. For this study you will have the following responsibilities:

1. Contact the president/chancellor’s office to verify the receipt of the questionnaire and obtain the name and contact information for the survey respondent.

2. Contact the survey respondent to verify receipt of the questionnaire and encourage the respondent to complete either the online version of the questionnaire or the paper copy. Record the questionnaire information over the phone only when it is absolutely necessary and when the respondent has ready access to the information requested.

3. Use your proficiency in the interviewing techniques of persuasion and refusal conversion.

4. Answer general questions about the purpose of the study, the way in which institutions were selected for inclusion, and how the data will be used. (See Frequently Asked Questions in Chapter 5). Questions you are not prepared to answer should be referred to your supervisor.

5. Arrange for emails and faxes (remails are a last option).

6. Record the result of each contact attempt on the call record.

7. Document refusals on the call record and complete a Non-Interview Report Form (NIRF) (Attachment 8).

8. Discuss problems or cases requiring special handling with your supervisor.

1.5 Answering Questions

Your training materials contain the answers to most questions that will arise during your calls. Suggested answers to the most frequently asked questions are listed in Chapter 5 of this manual and on page 7 of the questionnaire. This chapter of the manual will also be useful in answering questions. Please be familiar with the cover letter that is being sent to the president/chancellor’s office. You may use the information this letter contains to answer questions.

There will be times, however, when someone asks a question that you are not prepared to answer. You should never guess at the answer. Tell the respondent that you must check with your supervisor. Record the question and continue the interview if possible. Bring the case to the attention of your supervisor. It is important to respond to all questions or problems that arise during your telephone contact.

2. RESPONDENT INFORMATION SHEET (RIS)

You will be calling institutions that have not completed/submitted a survey. The Respondent Information Sheet (RIS) (Attachment 10) leads you through the introduction to the president/ chancellor’s office and the respondent, providing introductory remarks regarding the purpose of your call and the study. As the study progresses, you may adapt the introductory remarks to meet special circumstances.

2.1 Telephone Contact Steps

The purpose of the telephone contacts is to obtain the cooperation of institutions and prompt respondents to complete either the online or paper questionnaire. You may conduct an interview to record the questionnaire information by telephone only when necessary and when respondents have ready access to the information requested. Listed below are the steps you will follow to complete each telephone contact.

Step 1 Inspect your interviewer materials. Make sure that you have all materials necessary to complete the call. This includes a RIS with two labels, one with general institution information and a second with information for accessing the survey online.

Step 2 Contact the president/chancellor’s office. Verify that the person listed as the president is still there and that he or she has received the survey packet. Obtain the name and telephone number of the respondent to whom the survey was forwarded.

Step 3 Contact the respondent and gain his/her cooperation to complete the survey, preferably online, or to send it to Westat by fax.

Step 4 Record the results of the contact. Record the results of each contact or attempted contact on the call record. Also record any appointment information on the call record.

Step 5 Review and edit materials. After each contact or completed interview, review the RIS and the questionnaire. Read over any comments you have written to be sure that they are clear. Editing remarks or corrections on questionnaires should be made in blue pencil.

Step 6 Complete the Remail Request, Fax Request, or Email Request Form. If the president’s office or respondent has not received the questionnaire and requests that a copy be sent, complete a Remail Request Form (Attachment 5), Fax Request Form (Attachment 6), or Email Request Form (Attachment 7) and arrange to have the questionnaire completed and returned as soon as possible.

Step 7 Complete the NIRF for all refusals. Whenever an institution or designated respondent refuses to participate in the study, you must complete a Non-Interview Report Form (NIRF). The form (Attachment 8) provides for a detailed description of the refusal that is imperative for a refusal conversion attempt.

2.2 Overview of the RIS

The RIS (Attachment 10) is a guide for you to follow during your initial contact to the president/chancellor’s office and with the designated respondent. Most contacts will proceed in a manner that follows the RIS fairly closely. Below are step-by-step instructions for the use of this form. Please read the RIS as it is written until you become familiar with its contents. Special comments may be recorded on the RIS or call record.

The first round of telephone calls you make to the institutions will be prompt calls to encourage the institutions to quickly complete the questionnaire and return it to Westat. Remind respondents that an online version of the survey is available at www.nagb-survey.org and be prepared to offer the institution’s User ID and Password. The first step in this procedure is to contact the president/chancellor’s office at each institution. You will be checking with them to find out:

If the survey package was received;
Whether the questionnaire has been forwarded to a designated respondent and if Westat may contact the respondent;
What is the appropriate office(s) to complete our request, keeping in mind that we are looking for one respondent to coordinate completion of the questionnaire; and
The name and contact information for the designated respondent if our request has not been reviewed.

If the president/chancellor’s office would like Westat to follow up with the respondent, then the same steps outlined above will be followed with the respondent to ascertain the status of the questionnaire. If a questionnaire is to be remailed, faxed, or emailed to an institution, we would prefer to send the second copy of the questionnaire directly to the respondent, rather than to the president/chancellor’s office, because this pilot test must be conducted in six weeks. To have a questionnaire mailed to a respondent, complete a remail form. Be sure to legibly record the full name, mailing address, email address, or fax number of the respondent on the form. These forms go into their respective baskets to be handled by the clerical staff. In the interest of time, we would prefer to email the survey materials. You may send faxes and emails yourself. Emails will also be used to send follow-up reminders to the respondents in the third week.

2.3 Step-by-Step Instructions

2.3.1 President/Chancellor’s Office Contact

In the upper left corner of the RIS is a label that contains the following:

Institution ID - Each institution has an exclusive 6-digit identification number that will be used to identify the institution. This number should accompany any inquiries about a particular case. Institution IDs for two-year institutions will be the IPEDS number followed with an “02” and institution IDs for four-year institutions will be the IPEDS number followed with an “04.”
President/chancellor title.
Institution name and address.
Telephone number (usually the general telephone number of the institution).

Your initial contact will be to the office of the president or chancellor of the sampled institution. You will most likely reach the secretary to the president or chancellor, and you should introduce yourself and state the purpose of your call to that person. However, if you make direct contact with the president or chancellor, introduce yourself, explain that you are calling to follow up on a mailed survey, sponsored by the National Assessment Governing Board (NAGB).

Verify Package Receipt With Initial Contact

First, determine whether the president’s office received the survey package. It is helpful to mention that the packet contained a cover letter to the president/chancellor signed by Ray Fields, Assistant Director for Policy and Research of the National Assessment Governing Board, on National Assessment Governing Board letterhead (with a pink Web label for two-year institutions and a green Web label for four-year institutions, a seven-page questionnaire booklet on ivory-colored paper/yellow-colored paper, and a postage-paid return envelope.)

The next step is to encourage the institution’s participation and determine an appropriate respondent. If the president’s office received the survey packet, ask to whom the questionnaire was forwarded. Ask for the contact information for the designated respondent. These same steps will be followed with the designated respondent to ascertain the status of the questionnaire.

If the president/chancellor’s office is unsure about where our survey should be sent, you should provide some guidance. Because the survey asks about tests and test scores used to place students into developmental or remedial courses, many universities refer the survey to student services, the placement office, the math or English department, or the admissions office. Refer to page 3-3 in Chapter 3 for procedures for identifying the preferred survey respondent.

If the president/chancellor’s office has not yet reviewed our request or sent the survey package to a designated respondent, tell them that we will call back in a day or so to check on the status of our request. Explain the limited amount of time in which this pilot test must be completed (six weeks) and that it is important to get the survey to the appropriate respondent as soon as possible.

Survey Package Not Received

If the president/chancellor’s office has not received the survey package and requests a second copy of the survey be mailed, faxed, or emailed to the institution, inform him/her that because of our limited time to complete the survey, we would like to send the second copy of the questionnaire directly to the respondent. Explain that this is a pilot test and that we are trying to get all completed questionnaires within six weeks.

If the president/chancellor agrees, record the respondent information on the RIS and contact the designated respondent. To have a questionnaire mailed to the president’s office, complete a remail request form. Be sure to legibly record the full name, mailing address, email address, or fax number of the respondent on the form. This form goes into the respective basket to be handled by the clerical staff. You will then call the president’s office again to obtain the name of the designated respondent. If a fax or email is requested, you may send the email or fax yourself.

If you have the name of the designated respondent, contact that individual and inform him/her that you have been referred by the president/chancellor’s office (give name of contact). You will then encourage the institution’s participation, and remind the respondent that an online version of the survey is available at www.nagb-survey.org. Be prepared to offer the institution’s User ID and Password to the designated respondent. Remember to emphasize that this is a pilot test with a short turn-around time.

2.3.2. Describe process of finding appropriate respondent

One goal of the pilot study is to learn more about how easy or difficult it is to identify the best survey respondent through contact with the president or chancellor’s office. The first page of Attachment 10 is a form to be used for recording the process of identifying the survey respondent. For each institution you contact, please document this process using a blank Attachment 10 form. Include a full description of any problems encountered in finding the appropriate respondent and solutions given. For example, please describe whether the contact in the president or chancellor’s office was confused about the purpose of the study or if the contact was initially unsure who the questionnaire should be sent to. Please also describe situations where the contact in the president or chancellor’s office says that multiple personnel within the institution could be the appropriate respondent for all or part of the questionnaire.

2.3.3. Respondent Contact–Designated Respondent

Calls to respondents will proceed very much like the contacts you had with the president/chancellor’s office. When you reach a respondent, introduce yourself and explain the purpose of your call. Explain that the president’s office at the institution (give name of contact at the president/chancellor’s office) indicated that the questionnaire was forwarded to him/her, and that we are calling to check on the status of the questionnaire.

If the respondent indicates that the questionnaire has already been returned to Westat, record the date it was completed on the RIS and tell the respondent that we will look for the questionnaire. However, if we do not receive it online or within a week (if they have mailed it to us), we will call back to let them know. Ask the respondent if he/she kept a copy of the questionnaire and in the interest of time, if he/she could fax a copy of the completed questionnaire to us. Provide the respondent with the toll-free number (1-866-254-0984).

If the respondent is still working on the questionnaire, ask when we can expect to receive the questionnaire. Explain that the study is a pilot test and that we are looking to receive all completed questionnaires as soon as possible. Record the estimated completion date on the RIS. Remind the respondent that the survey is designed to be completed online at www.nagb-survey.org. Be prepared to offer the institution’s User ID and Password.

If the respondent has not received the questionnaire from the president/chancellor’s office, inform him/her that the survey can be completed at www.nagb-survey.org. Offer the institution’s User ID and Password. If the respondent prefers, a copy can be sent to him/her by email or by fax. You may send these emails/faxes yourself.

If the designated respondent has given the questionnaire to someone else to complete, ask if we can contact the person completing the questionnaire directly to determine the status of the questionnaire. If the designated respondent agrees to this, record the new designated respondent information on the RIS and immediately call the new designated respondent. Follow the same steps as the respondent contact.

3. QUESTIONNAIRE ADMINISTRATION

This chapter presents instructions for questionnaire administration. You will need to be familiar with the questionnaire so that you can assist respondents if necessary.

3.1 General Instructions on the Use of the Questionnaire

As previously indicated, the survey package was mailed on August 2, 2010. Separate versions of the questionnaire were mailed to two-year and four-year institutions. The first round of calls you make to the institutions will be prompt calls to encourage the institutions to quickly complete the questionnaire online or by mail, email, or fax.

3.2 Overview of the Questionnaire

The questionnaire consists of a cover page, a page with survey definitions and instructions, four pages of questions, and a page with frequently asked questions (FAQ). The survey questions are divided into two sections: one for mathematics tests and one for reading tests. There are four questions on mathematics tests and four questions on reading tests. The questions in the reading section are similar to those in the mathematics section with the exception of the tests listed.

There are two versions of the questionnaire: one for two-year institutions and one for four-year institutions. The questionnaires differ only in the instructions given for reporting test scores. Respondents from two-year institutions are instructed to report based on the scores used to evaluate students who are enrolled in a degree program that is designed to transfer to a four-year institution. These programs vary from institution to institution but typically focus on general education and result in an associate’s degree (AA). Programs at two-year institutions that are not designed to transfer to four-year institutions are often intended to prepare a student for a particular career, such as nursing.

Respondents from four-year institutions are instructed to report based on the scores used to evaluate students who are enrolled in a liberal arts and sciences undergraduate degree program. This instruction is intended to convey that we are interested in the scores used to evaluate the general entering student population. For example, institutions should report tests and scores used to evaluate entering students enrolled in a program that awards a bachelor of arts degree (BA) or bachelor of science degree (BS).

The cover page contains various useful pieces of information, including the survey definition, the title of the survey, the name of the sponsoring agency, the OMB clearance number, and a statement indicating that the survey is voluntary. In addition, the cover page of the questionnaire provides Westat’s address, toll-free telephone and fax numbers, and the name and e-mail address of the survey manager to whom questions can be referred. A label was affixed to each questionnaire listing school information, including the IPEDS ID number, the school name, and address. The cover page also asks for the respondent’s name, title and contact information, and an estimate of the number of minutes taken to respond to the survey. Note that on the web survey, respondents are asked about the time it took to complete the survey at the end of the questionnaire, rather than at the beginning as it is on the paper-and-pencil version.

The second page of the questionnaire provides an overview of the survey as well as important definitions and instructions intended to assist respondents in completing the survey. The second paragraph (bolded text) is different for two-year and four-year institutions. Otherwise the pages are identical for both versions of the questionnaire.

The third through seventh pages of the questionnaire contain the survey questions and the FAQ page. These are discussed in more detail later in this manual.

3.3 Eligibility and Identifying the Preferred Survey Respondent

Eligible Institutions. Institutional eligibility is different for two-year and four-year institutions:

Two-year institutions are eligible if they had any students enrolled in a degree program designed to transfer to a four-year institution in fall 2009. Any two-year institution that meets this criterion is eligible to complete the survey, including public, private not-for-profit, and private for-profit institutions.
Four-year institutions are eligible if they had any students enrolled in an undergraduate liberal arts and sciences degree program in fall 2009. Any four-year institution that meets this criterion is eligible to complete the survey, including public, private not-for-profit, and private for-profit institutions.

If you identify an institution that appears to be ineligible, please complete an interviewer problem sheet and include a full explanation of the problem.

Identifying the Preferred Survey Respondent. The survey package with the cover letter and questionnaire was sent to the office of the president or chancellor of each sampled institution. The preferred survey respondent, however, is likely to be in another location within the institution. You will contact the president’s office to determine who should complete the survey and then contact the individual identified as the appropriate respondent by the president’s office.

The survey cover letter asked the office of the president or chancellor to direct the questionnaire to the office or individual within the institution that is most knowledgeable about the institution’s policy for evaluating entering students’ need for developmental or remedial courses in mathematics and reading. We determined during survey development that the location of the appropriate survey respondent can vary greatly from institution to institution. Some possible locations of the appropriate survey respondent are as follows:

Offices of academic deans
Offices that provide student services
Offices that oversee new students
Offices of admissions
Academic departments, such as mathematics or English departments

In your communication with the office of the president or chancellor, please emphasize that we are interested in the person or office that is responsible for evaluating student need for developmental or remedial courses. We are not interested in student enrollment in developmental or remedial courses per se, but rather in the test scores used by the institution to determine whether or not students need remediation in mathematics or reading. If the contact in the office of the president or chancellor seems confused about this, it may help to refer to some of the tests we are interested in:

ACCUPLACER
COMPASS
ASSET
ACT or SAT tests used to evaluate entering students’ need for developmental or remedial courses (rather than for making admissions decisions)
Tests developed by the institution or within the institution’s home state that are used to evaluate entering students’ need for developmental or remedial courses.

In some cases, there may be multiple individuals or offices within the institution that would be appropriate survey respondents. One potential scenario would be different academic departments that separately handle the evaluation of entering students for developmental or remedial courses. For example, the mathematics department may handle evaluation/testing in mathematics while the English department handles evaluation/testing in reading. If you encounter this or another situation with multiple potential respondents, please ask the contact in the office of the president or chancellor if there is an administrative office that oversees the potential respondents and would be willing to coordinate the institution’s survey response by collecting the data from the different staff or offices and completing the questionnaire. If this is not possible, ask if one of the potential respondents would be willing to coordinate the survey response instead. The goal is to have one respondent at each institution who is responsible for coordinating the response and completing the questionnaire, even if he or she needs to seek out data from other personnel or offices.

Please remember to describe the process of finding the appropriate respondent using the form included on the first page of Attachment 10 (RIS).

3.4 Question by Question Specifications

This section provides additional detail on the instructions/definitions page and the survey questions.

3.4.1 Instructions and Definitions Page

The second page of the questionnaire lists several important instructions and definitions intended to assist respondents in completing the survey.

Focus of the Survey. The first paragraph (shown below) appears on both the two-year and four-year versions of the questionnaire and is intended to introduce the respondent to the focus of the survey—the tests used to evaluate whether entering students are in need of developmental or remedial courses in mathematics or reading. Examples of tests that may be used for this purpose are given in the bullets.

This survey examines the test scores used by postsecondary institutions to evaluate whether entering students are academically prepared or in need of developmental or remedial courses in mathematics or reading. The goal of the survey is to identify the test scores below which students are deemed in need of developmental or remedial courses (i.e., academically unprepared for entry-level courses). The scores may come from a variety of tests, including:

Admissions tests such as the ACT or SAT;
Placement tests such as ACCUPLACER, ASSET, and COMPASS; or
Institution- or state-developed tests.

Reporting Instructions for Two-Year Institutions. The instruction below appears on the questionnaire for two-year institutions:

Please report based on the tests your institution uses to evaluate entering students who are pursuing a degree program that is designed to transfer to a four-year institution.

Some two-year institutions use different tests or other evaluation criteria depending on a student’s program of enrollment. We are interested in the tests used for students who are enrolled in a degree program that will allow them to transfer to a four-year institution. These programs vary from institution to institution but typically focus on general education and result in an associate’s degree (AA). Programs at two-year institutions that are not designed to transfer to four-year institutions are often intended to prepare a student for a particular career, such as nursing.

Reporting Instructions for Four-Year Institutions. The instruction below appears on the questionnaire for four-year institutions:

Please report based on the tests your institution uses to evaluate entering students who are enrolled in an undergraduate degree program in the liberal arts and sciences.

Like two-year institutions, some four-year institutions use different evaluation criteria depending on the student’s program of enrollment. This instruction is intended to convey that we are interested in the scores used to evaluate the general student population. For example, scores used to evaluate entering students who are enrolled in a bachelor of arts (BA) degree program or bachelor of science (BS) degree program.

General Reporting Instructions for Both Two-Year and Four-Year Institutions. The following paragraph appears on both versions of the questionnaire:

Answer all relevant sections, even if your institution does not offer developmental or remedial courses or offers only one or two such courses. Use data from your institutional records whenever possible. If exact data are not available, then give your best estimate.

The first sentence is intended to convey that any respondent that uses tests or other criteria to evaluate entering students for developmental or remedial courses in mathematics or reading should complete applicable questionnaire sections, even if they do not offer developmental or remedial courses (or offer only a few courses). In survey development, we encountered some respondents that offered only one or two developmental or remedial courses who were unsure if they should report the evaluation criteria for those courses. The second and third sentences instruct respondents to report based on actual records whenever possible, but best estimates are acceptable if records are not available.

Definition of Developmental or Remedial Courses. The definition of developmental and remedial courses appears on both versions of the questionnaire:

Developmental or remedial courses are generally designed to improve the skills of entering students who are not academically prepared for entry-level college courses, as determined by your institution. This survey focuses on developmental or remedial courses that address skills in mathematics and reading. Developmental or remedial writing courses should not be considered unless they have a substantial focus on improving reading skills.

The definition is intentionally open-ended, as these courses can be quite different from institution to institution. However, the key feature we are looking for in developmental or remedial courses is that they are meant to prepare entering students for entry-level college courses (see the definition of entry-level courses on the following page). Additionally, for this survey respondents should only consider evaluation for remedial and developmental courses that address mathematics or reading skills. Evaluation for remedial or developmental courses that address other subjects, such as foreign language or science courses, is not to be considered.

The last sentence in the definition above is a clarification about how to handle writing courses. During survey development, some potential respondents indicated that developmental or remedial writing courses can have a focus on improving reading skills. Respondents should report scores used for writing courses only if the course has a substantial focus on improving reading skills (a good rule of thumb is that at least half of the course content is focused on reading skills).

The following bulleted notes appear below the definition of developmental or remedial courses:

Please note:

In most cases, developmental or remedial courses are not credit-bearing and they do not count toward general education or degree requirements.
Your institution may use other names such as “preparatory,” "compensatory," or "basic skills" or some other term to refer to developmental or remedial courses.

These notes are clarifications of the definition of developmental and remedial courses. The first bullet indicates that in most cases students do not earn credit for developmental or remedial courses, although in survey development we encountered a few institutions that do award credit for these courses. The second bullet refers to the various names institutions may use to refer to developmental or remedial courses. Many institutions seem to have moved away from the term “remedial” in favor of “developmental” or some other term.

Again, the key feature of developmental or remedial courses is that they aim to prepare entering students for entry-level college courses and focus on improving mathematics and reading skills. Respondents should report tests used to evaluate students need for such courses, even if the course happens to be credit-bearing or has a title that doesn’t include “remedial” or “developmental.”

Definition of Entry-Level College Courses. The definition of entry-level college courses contains a number of important points respondents need to be aware of. This definition appears on both versions of the questionnaire:

Entry-level college courses refer to first-year credit-bearing courses that require college-level mathematics or reading skills, as these skills are defined by your institution. Entry-level college courses typically count toward general education or degree requirements. Please note that a variety of entry-level courses may require college-level mathematics or reading skills, including entry-level humanities, mathematics, and science courses.

For this survey, entry-level college courses refer to courses that are typically taken by entering college students and that require college-level mathematics and reading skills (as defined by the institution). Entry-level college courses are credit-bearing and generally count toward a student’s degree requirements. It is important to note that a variety of entry-level courses fall under this definition, not just entry-level mathematics or English courses. The last sentence of the definition is meant to clarify this for respondents.

Definition of Entering Students. The final definition on the second page covers “entering students.” This definition appears on both versions of the questionnaire:

Entering students include full-time and part-time students who are new to your institution and are subject to your institution’s policy for determining need for developmental or remedial courses.

The key part of the definition is that entering students are subject to evaluation under the institution’s policy for determining need for developmental or remedial courses. Some students who are new to an institution, such as transfer students, may or may not be subject to the institution’s evaluation policy. For example, an institution may evaluate transfer students entering on the second-year level but not evaluate students entering on the third- or fourth-year level. An institution might also evaluate some entering students and not others depending on the student’s program of enrollment. We want respondents to consider only those entering students who are subject to evaluation.

3.4.2. Survey Questions

The remainder of this section discusses each of the survey questions. Note that the questions in Section B (reading tests, questions 5 through 8) are very similar to those in Section A (mathematics tests, questions 1 through 4), with the exception of the names of the tests given.

Directions at top of page 3. This is a filter item intended to screen out institutions that do not meet the survey’s basic eligibility criterion. The text differs for two-year and four-year institutions:

Two-year institutions: Directions: If your institution did not have any entering students in fall 2009 who were pursuing a degree program designed to transfer to a four-year institution, please check this box , complete the cover page, and return the questionnaire.

Four-year institutions: Directions: If your institution did not have any entering students in fall 2009 who were enrolled in an undergraduate degree program in the liberal arts and sciences, please check this box , complete the cover page, and return the questionnaire.

Respondents who check the box are ineligible for the survey but should still fill out the information requested on the cover page and return the questionnaire to Westat. If you are obtaining the information over the telephone or otherwise discussing this direction with the respondent, be sure to confirm that his/her institution did not enroll the type of student specified in fall 2009. If the respondent’s institution did enroll the type of student specified in fall 2009, he/she should proceed through the questionnaire, answering all relevant sections.

Question 1. This is a filter question to determine if respondents used any test in fall 2009 to evaluate entering students for developmental or remedial mathematics courses. Several of the most common “off-the-shelf” placement tests are listed in the question stem, but respondents should answer “yes” if any test was used for this purpose, even if the test was developed internally by the institution or within the institution’s home state. For example, public colleges and universities in Washington State use a state-developed unified system of testing called the Academic Placement Testing Program to evaluate entering students. Respondents should skip to question 3 if they answer “no.”

Two instructional bullets appear below the question stem. The first bullet varies depending on the institution type:

Two-year institutions: Consider tests used to evaluate entering students who were pursuing a degree program designed to transfer to a four-year institution.

Four-year institutions: Consider tests used to evaluate entering students who were enrolled in an undergraduate degree program in the liberal arts and sciences.

The second bullet is intended to instruct respondents to report any evaluation performed, even if the institution does not offer developmental or remedial courses (or offers only a few courses). This instruction has a similar purpose to the instruction in the third paragraph of the definitions and instructions page.

Question 2. This question asks for the tests and scores used to evaluate whether entering students were in need of developmental or remedial mathematics courses in fall 2009. Respondents should skip this question if they answered “no” to question 1. This question is the same for both the two-year and four-year versions of the survey.

There are two instructional bullets on question 2. The first bullet address institutions that use one score to recommend students for developmental or remedial courses and another score to require students for these courses. In survey development, we found some institutions that have different scores to recommend or require students for developmental or remedial courses. This seemed to be most common when the institution required students who were definitely not prepared for entry-level courses and recommended students for developmental or remedial courses if they were on the borderline of prepared and unprepared. Respondents should report the highest score used if their institution uses different scores to require or recommend students for remedial or developmental courses.

The second bullet refers to institutions that have multiple levels of developmental or remedial courses that use different scores as selection criteria. For example, some institutions may offer a low-level basic algebra developmental course and an intermediate-level algebra developmental course. Institutions with this arrangement may use a lower score to identify students for the basic course and a higher score for the intermediate course. Respondents should report the highest score used if this situation applies to them.

Column A lists a number of different tests respondents may use to evaluate entering students for remedial or developmental mathematics courses. The score ranges for each test are given in parentheses next to the test name. For each test that respondents used in fall 2009, they should check the box in column B next to the test name and enter the score in column C below which students were deemed to need developmental or remedial mathematics courses. Respondents should report only one score for each test used, not a range of scores (e.g., 16–24). If respondents indicate that they would prefer to report a range of scores, instruct them to report only the highest score and describe their scoring approach for the test in more detail in the comment box. The response in column C must be a number, not text. If respondents used tests that are not listed in column A, they should write in the names in rows t through v. These tests could be developed by the institution or state. Scores are not necessary for these rows.

It is important to remember that the score we ask for—the score below which students are in need of developmental or remedial courses—is also in many cases the score above which students are prepared for entry-level college courses. The National Assessment Governing Board is interested in determining the score point that separates prepared students from unprepared students, so this is an important concept to keep in mind while communicating with respondents. In some cases respondents may refer to this as the “cut score,” or the score that is used as a dividing line to distinguish prepared students from unprepared students.

Several other items on question 2 are worth noting in more detail. The list of tests in column A includes ACT and SAT tests. Scores from these tests are commonly used for making admissions decisions, but they are also often used to determine if entering students are in need of developmental or remedial courses. If a respondent is confused about what score to report for ACT or SAT, emphasize that only the score below which an entering student is deemed in need of developmental or remedial mathematics courses should be reported. For example, remind the respondent that we are only interested in scores that can result in placement into developmental or remedial mathematics or reading courses. We have also listed ACT composite and SAT total scores, which were found to be used by some institutions during survey development. Institutions are also given the options to report the SAT total score both with the SAT writing component and without the writing component.

Some respondents may say that they cannot report a single score for a given test because the score can vary depending on some other factor. In survey development we encountered some institutions that use different scores to identify students for developmental or remedial courses depending on high school grades. For example, one institution told us that the cut score used to identify students as in need of developmental mathematics courses was lower for students with good grades in high school mathematics and higher for students with poor grades in high school mathematics. In other words, the institution set a higher bar for students with poor high school mathematics grades to make sure that they were prepared for entry-level college coursework. Respondents that have this or a similar situation should be instructed to report the highest score that can result in a student being referred to a developmental or remedial course, and they could provide more details about their policies in the comment box.

A comment box is provided below question 2. Respondents can use it to enter any comment they wish to make about their response to question 2. For example, some respondents may wish to clarify how they use a particular test in evaluating students for developmental or remedial mathematics courses. This box is limited to 500 characters on the web survey.

Question 3. This is a filter question intended to determine if respondents used any other evaluation method (aside from the tests listed in question 2) to determine if entering students were in need of developmental or remedial mathematics courses in fall 2009. A bullet provides examples of the kinds of other criteria respondents should consider while answering the question. Descriptions of each example are as follows:

High school graduation exams refer to tests given to graduating high school students to assess general preparedness and mastery of skills expected of high school graduates. These include state-administered tests such as the New York Regents Exam on which students must score above a specified point to receive a high school diploma.
End-of-course exams refer to tests given to high school students to assess mastery of a particular academic subject, such as algebra or English.
High school records refer to high school grades, grade point average, certificates, or other academic records maintained by high schools.
Advanced Placement or International Baccalaureate exams are tests given as part of a student’s enrollment in either the AP or IB programs.
Faculty recommendations refer to recommendations written for an entering student by a high school or college faculty member.

Respondents that select “yes” should proceed to question 4. Respondents that select “no” should skip to question 5.

Question 4. This question asks respondents to report the other criteria used to evaluate students for developmental or remedial mathematics courses (aside from the tests listed in question 2). Respondents should check the box for each criterion used in fall 2009. If a respondent used an evaluation criterion not listed, he/she should check the box in row f and write in the criterion in the space provided. Descriptions of each criterion are as follows:

High school graduation exams refer to tests given to graduating high school students to assess general preparedness and mastery of skills expected of high school graduates. These include state-administered tests such as the New York Regents Exam on which students must score above a specified point to receive a high school diploma.
End-of-course exams refer to tests given to high school students to assess mastery of a particular academic subject, such as algebra or English.
High school grades refer to high school grades or high school grade point average.
Highest high school mathematics course completed refers to the highest level of high school mathematics completed by an entering college student.
Advanced Placement or International Baccalaureate exams are tests given as part of a student’s enrollment in either the AP or IB programs.
Faculty recommendations refer to recommendations written for an entering student by a high school or college faculty member.

A comment box appears below question 4. Respondents can enter any comments about their response to question 4 in the comment box. For example, respondents may wish to elaborate on how they use the other evaluation criteria noted in question 4, or provide more detail on what criteria they use. This box is limited to 500 characters on the web survey.

Instruction above question 5. The following instruction appears above question 5:

Please note: Developmental or remedial writing courses should not be considered reading courses unless they have a substantial focus on improving reading skills.

This instruction is a clarification about how to handle writing courses. During survey development, some potential respondents indicated that developmental or remedial writing courses can have a focus on improving reading skills. Respondents should report scores used for writing courses only if the course has a substantial focus on improving reading skills (a good rule of thumb is that at least half of the course content is focused on reading skills).

Question 5. This is a filter question to determine if respondents used any test in fall 2009, to evaluate entering students for developmental or remedial reading courses. Several of the most common “off-the-shelf” placement tests are listed in the question stem, but respondents should answer “yes” if any test was used for this purpose, even if it was developed internally by the institution or within the institution’s home state. For example, public colleges and universities in Washington State use a state-developed unified system of testing called the Academic Placement Testing Program to evaluate entering students. Respondents should skip to question 7 if they answered “no.”

Two instructional bullets appear below the question stem. The first bullet varies depending on the institution type:

Two-year institutions: Consider tests used to evaluate entering students who were pursuing a degree program designed to transfer to a four-year institution.

Four-year institutions: Consider tests used to evaluate entering students who were enrolled in an undergraduate degree program in the liberal arts and sciences.

Question 6. This question asks for the tests and scores used to evaluate whether entering students were in need of developmental or remedial reading courses in fall 2009. Respondents should skip this question if they answered “no” to question 5. This question is the same for both the two-year and four-year versions of the survey.

There are two instructional bullets on question 6. The first bullet address institutions that use one score to recommend students for developmental or remedial courses and another score to require students for these courses. In survey development, we found some institutions that have different scores to recommend or require students for developmental or remedial courses. This seemed to be most common when the institution required students who were definitely not prepared for entry-level courses and recommended students for developmental or remedial courses if they were on the borderline of prepared and unprepared. Respondents should report the highest score used if their institution uses different scores to require or recommend students for remedial or developmental courses.

The second bullet refers to institutions that have multiple levels of developmental or remedial courses that use different scores as selection criteria. For example, some institutions may offer a low-level developmental reading course for students with low reading skills and an intermediate-level developmental reading course for students whose reading skills are slightly higher but not yet up to the level needed for entry level college coursework. Institutions with this arrangement may use a lower score to identify students for the low-level course and a higher score for the intermediate course. Respondents should report the highest score used if this situation applies to them.

Column A lists a number of different tests respondents may use to evaluate entering students for remedial or developmental reading courses. The score ranges for each test are given in parentheses next to the test name. For each test that respondents used in fall 2009, they should check the box in column B next to the test name and enter the score in column C below which students were deemed to need developmental or remedial reading courses. Respondents should report only one score for each test used, not a range of scores (e.g., 16–24). If respondents indicate that they would prefer to report a range of scores, instruct them to report only the highest score and describe their scoring approach for the test in more detail in the comment box. The response in column C must be a number, not text. If respondents used tests that are not listed in column A, they should write in the names in rows t through v. Scores are not necessary for these rows.

Several other items on question 6 are worth noting in more detail. The list of tests in column A includes ACT and SAT tests. Scores from these tests are commonly used for making admissions decisions, but are also often used to determine if entering students are in need of developmental or remedial courses. If a respondent is confused about what score to report for ACT or SAT, emphasize that only the score below which an entering student is deemed in need of developmental or remedial mathematics courses should be reported. We have also listed ACT composite and SAT total scores, which were found to be used by some institutions during survey development. Options to report the SAT total score both with the SAT writing component and without the writing component are given.

Some respondents may say that they cannot report a single score for a given test because the score can vary depending on some other factor. For example, in survey development we encountered some institutions that use different scores to identify students for developmental or remedial courses depending on high school grades. Respondents that have this or a similar situation should be instructed to report the highest score that can result in a student being referred to a developmental or remedial course and provide more details about their policies in the comment box.

A comment box is provided below question 6. Respondents can use it to enter any comment they wish to make about their response to question 6. For example, some respondents may wish to clarify how they use a particular test in evaluating students for developmental or remedial reading courses. This box is limited to 500 characters on the web survey.

Question 7. This is a filter question intended to determine if respondents used any other evaluation method (aside from the tests listed in question 6) to determine if entering students were in need of developmental or remedial reading courses in fall 2009. A bullet provides examples of the kinds of other criteria respondents should consider while answering the question. Descriptions of each example are as follows:

High school graduation exams refer to tests given to graduating high school students to assess general preparedness and mastery of skills expected of high school graduates. These include state-administered tests such as the New York Regents Exam on which students must score above a specified point to receive a high school diploma.
End-of-course exams refer to tests given to high school students to assess mastery of a particular academic subject, such as algebra or English.
High school records refer to high school grades, grade point average, certificates, or other academic records maintained by high schools.
Advanced Placement or International Baccalaureate exams are tests given as part of a student’s enrollment in either the AP or IB programs.
Faculty recommendations refer to recommendations written for an entering student by a high school or college faculty member.

Respondents that select “yes” should proceed to question 8. Respondents that select “no” should stop at this point, complete the information requested on the cover page, and return the questionnaire to Westat.

Question 8. This question asks respondents to report the other criteria used to evaluate students for developmental or remedial reading courses (aside from the tests listed in question 6). Respondents should check the box for each criterion used in fall 2009. If a respondent used an evaluation criterion not listed, he/she should check the box in row f and write in the criterion in the space provided. Descriptions of each criterion are as follows:

High school graduation exams refer to tests given to graduating high school students to assess general preparedness and mastery of skills expected of high school graduates. These include state-administered tests such as the New York Regents Exam on which students must score above a specified point to receive a high school diploma.
End-of-course exams refer to tests given to high school students to assess mastery of a particular academic subject, such as algebra or English.
High school grades refer to high school grades or high school grade point average.
Highest high school English course completed refers to the highest level of high school English completed by an entering college student.
Advanced Placement or International Baccalaureate exams are tests given as part of a student’s enrollment in either the AP or IB programs.
Faculty recommendations refer to recommendations written for an entering student by a high school or college faculty member.

A comment box appears below question 8. Respondents can use it to enter any comments about their response to question 8. For example, respondents may wish to elaborate on how they use the other evaluation criteria noted in question 8, or provide more detail on what criteria they use. This box is limited to 500 characters on the web survey.

4. CASE MANAGEMENT AND PROCEDURES

Staff assigned to the data collection task will coordinate their activities to effectively manage and properly document the outcomes of their efforts. The interviewers will be responsible for prompting institution respondents, resolving questions that the respondents might have about the meaning and intent of the survey questions, and responding to other questions that the respondents might have. The data collection task leader will oversee all survey operations, review the status of each case, and work with the staff to ensure that appropriate steps are taken to maximize data quality. Documentation and file organization needs to be standard, so that all staff picking up on the work can readily understand what occurred during previous calls.

4.1 Receiving and Managing Cases

Your supervisor will assign cases to you. Your cases should be arranged on your desk so that project staff has easy access to them. Do not keep your cases in locked drawers at the end of the workday. New work should not be removed from the baskets, but you should not sit around waiting for new work. If you are running low on work, please let your supervisor know and new cases will be assigned to you.

Organize your survey materials in a manner that is useful and efficient to you. It is a good idea to organize your cases before you start each day. Create a "to be worked pile" and organize it by time zones. As you begin each day, give first priority to specific appointments scheduled, then to new work, and finally to interim cases that have been worked previously.

Phone call attempts should be as productive as possible. The goal is to finalize a case in as few attempts as possible. For example, if the previous call placed for a case was made in the early morning, with a "ring no answer," then the next call should be placed later in the morning or sometime in the afternoon. Generally, calls should not be duplicated on a succeeding day in time slots similar to previous, unsuccessful attempts.

Keeping clear, legible documentation of every phone class you make is an important part of the research process. By doing so, we can treat each respondent according to his/her needs and monitor the productivity of the study. Your supervisor and other project staff will monitor the needs of the active files, such as those cases that require additional calls or special handling. Accurate and timely communication between the interviewing staff and the project staff is vital.

4.2 Setting Telephone Appointments

Appointments can be set during the hours the study will be in operation. At this time, the study's hours of operation will be Monday through Friday, 8:00 a.m. to 6:30 p.m., Eastern time. Telephone calls will be made to all time zones. When placing calls, be sure that you are calling at appropriate times.

4.3 Closing Off Your Work Day

At the end of your workday, please follow the guidelines below:

Check each completed case to be sure you have asked all items on the RIS and/or questionnaire and that you have clearly recorded all the responses.
Attach completed surveys to the RIS and place them in the appropriate basket.
Place problem cases in the problem basket for your supervisor to review.
Place Remail Request Forms, Fax Request Forms and Email Request Forms in the appropriate baskets.
Check your work before you leave to make sure you have taken care of everything you need to attend to that day.

4.4 Overview of the Call Record

A call record (Attachment 11) will be generated for each institution that you are to contact. You will find the ID number of the case in the lower right portion of the call record. The call record serves a twofold purpose: (1) it serves as your work assignment by providing information that you will need regarding the case that you are to call; and (2) it is the only means by which you communicate the status of each telephone call you attempt. Always review the call record and any previous comments written by you or other interviewers.

After you contact or attempt to contact an institution, you must record the result on the call record. Below is an explanation of items you will be using on the call record.

Interviewer Name: Record your first initial and last name for every contact or attempt to contact that you make.
Date: Record the month and day of every contact or attempt, e.g., 08/11.
Time Began: Record the time you called/attempted to call and indicate a.m. or p.m.
Time Ended: Record the time you finished the call and indicate a.m. or p.m. If the call does not result in contact, place a dash (-) in this space.
Result: Record the result code of the call/interview by using one of the codes listed in the result codes section of the call record (bottom portion). Result codes are further discussed in the next two sections.
Comments: Record explanations or notes regarding the telephone number, the interview, or the respondent. This is also the place to indicate when an institution should be called again if the questionnaire was not received. This column is the only means by which you can communicate with your colleagues. Limit your comments to one line per call if possible. However, if you feel additional explanation is necessary, attach a note to the call record. If you attach a note, remember to record the appropriate ID number, date, and your initials on the note. If you are running out of space on the call record, please attach another blank call record to the case. Please be sure to record the appropriate case ID number, institution name, and state on the new call record.
Callback Information: If a specific appointment is made with a respondent to complete the interview, record that information in the designated columns. Record the month and day (e.g., 09/25) and the time (indicate a.m. or p.m.). Always record the time first in the respondent's time, and convert it to Westat time outside the box. Please designate the conversion by using the code "EDT" or "EST."
Case Identification Number: The case ID number will appear on the bottom right of the call record. This ID number must match the case ID on the RIS.

4.5. Interim Result Codes

All interim codes are to be recorded as numbers. They are used only when the outcome of the contact does not result in a final disposition.

(1) No Answer: Use code "1" when no one answers the telephone when you call. It is important that you let the telephone ring no more than six times.

(2) First Refusal/Breakoff: Use code "2" when a respondent refuses to participate in the survey or breaks off the contact before you are able to complete the interview. (This is an interim code because the case will be reassigned during the nonresponse conversion effort.) You must complete a NIRF for any case receiving a code "2." Record code “2” on the call record and place the case in the refusal basket.

(3) Busy Signal: Use code "3" when the number is busy. If you get a busy signal, someone is usually at the number so try again in 10 minutes. All busy signals must be attempted twice during your work period.

(4) Callback - No Appointment: Use code "4" when you call a number but cannot complete the interview and the person you talk to does not give you a specific time or day to call back. You may use this code (a) when you have completed part of the interview but must call back to complete the remainder, or (b) when a respondent is not available or cannot do the interview during this call and does not give a specific callback appointment. If the respondent prefers to be called back at a general time of day (e.g., early a.m. or after lunch), note this in the comments. This code is also used when a respondent indicates that he/she will be mailing the questionnaire back on a certain date. Indicate in the comments section that the questionnaire will be mailed back, and give the expected mailing date. Information must be written in the comments and callback information sections whenever a code "4" is used. Note: A code "5" (discussed next) is always preferable to a code "4."

Code "4" should always have a callback date and time.

(5) Callback - Appointment: Use code "5" when you call a number and arrange a specific day and time to call back to talk to a respondent (e.g., Monday at 10:30 a.m., Wednesday at 5:00 p.m., etc.). Remember to convert the appointment time to Westat time in the right hand margin and designate EDT/EST. Whenever a "5" is recorded, information must be written in the comment and callback information space of the call record. You will use this code when you have been given a specific time to call back by either the secretary or the respondent. You should note on the call record the person with whom you arranged the appointment.

Code "5" should always have a callback date and time.

(7) Survey mailed: Use code "7" when the respondent has already mailed the questionnaire back to Westat. Whenever a "7" is recorded, information must be written in the comments and callback information space of the call record. Indicate that the questionnaire has already been mailed back, and give the mailing date.

Code "7" should always have a callback date.

(8) Problem/Other: Use code "8" anytime there is a problem with the interview and no other code is appropriate. Record what the problem is in the comment column and on an interviewer problem sheet (Attachment 9). These cases should be placed in the problem basket for review by your supervisor. Do not keep problem cases on your desk.

(9) Mailout Needed: Use code "9" when a remail is needed before the questionnaire is completed. Whenever a "9" is recorded, information must be written in the comments and callback information space of the call record. Indicate that the questionnaire has been remailed. Be sure to complete a remail request form.

Code "9" should always have a callback date.

(10) Tracing Needed: Not used. Use code "8" (problem) instead.

(11) Left 800 number: Use code "11" to indicate that the 800 number has been left with a respondent. Even when the 800 number is left, you must follow up with the respondent if a call is not returned within a reasonable time. For this study, it’s two to three days. Please indicate the date the 800 number was left on the call record.

Code "11" should always share a callback date.

Fax needed: Use code "12" whenever a respondent requests that a copy of the questionnaire be faxed to him/her. Make an appointment to call back to take the information by telephone; record this information on the call record.

Code "12" should always share a callback date.

Email needed: Use code "13" whenever a respondent requests that a questionnaire or Web Information Sheet be emailed to him/her. Make an appointment to call back to take the information by telephone; record this information on the call record.

Code "13" should always share a callback date.

Reminder: Codes 4, 5, 7, 9, 11, 12, and 13 should have callback dates.

4.6. Final Result Codes

All final codes are to be coded as letters and should be used whenever an interview is finalized. Not all of the codes listed on the call record will be used. The codes that will be used are explained below.

(CP) Completed by phone: Code "CP" when you have completed the entire interview by phone. A complete means all information has been obtained from the respondent, including a fully completed questionnaire.

(CF) Completed by fax: Code “CF” is used when you receive a completed questionnaire by fax. You are responsible for editing your CFs.

(RB) Final Refusal/Breakoff - This code will only be assigned by a refusal conversion interviewer or the supervisor and must be initialed by the survey manager.

(S1) Received by Mail/Web/Email: Interviewers will assign this code only when directed to do so by a supervisor. It is used when a completed questionnaire is received by mail, web, or email, and the case is being removed from telephone follow-up.

(MC) Maximum Contact: A supervisor will assign this code when the case needs to be closed out.

5. SUGGESTED ANSWERS TO FREQUENTLY ASKED
QUESTIONS AND OBJECTIONS

5.1 Questions About Survey Content

Note: Respondents with questions about the following issues that cannot be resolved with information in this training manual can be referred to the survey manager:

Academic programs with different criteria for evaluating students
A particular course that might or might not be a developmental/remedial course
A particular course that might or might not be an entry-level course
Questions about eligibility
Any other difficult-to-resolve issue that needs to be brought to the attention of the survey manager

My institution does not use placement tests.

Although tests are the main focus of the survey, there are two questions that ask about other ways institutions evaluate entering students, such as the use of high school grades. We need your response to these and any other questions that apply to your institution.

My institution does not offer developmental or remedial courses.

We are interested in how institutions evaluate entering students for developmental or remedial courses. Some institutions that do not offer developmental or remedial courses still evaluate student need for these courses. We need your response regarding any such evaluation performed by your institution.

My institution does not use the term “developmental courses” or “remedial courses.”

That is fine. We know that different institutions use different names for courses designed to address the academic deficiencies of entering students to prepare them for credit-bearing college-level courses. Please report the criteria used by your institution to evaluate entering students for these courses, even if they are not called “developmental” or “remedial” courses.

My institution does not do ANY evaluation of entering students for developmental or remedial courses in either mathematics or reading.

That is fine. However, in order for the survey results to be representative, we still need your response to questions that allow you to respond “yes” or “no” (Questions 1, 3, 5, and 7). Please complete these questions and submit the questionnaire.

My institution does not offer developmental/remedial reading courses, but it does offer developmental/remedial writing. What should I do?

Please report on the evaluation criteria for developmental or remedial writing classes only if the class has a substantial focus on improving reading skills. At least half of the course content should focus on improving reading skills.

My institution uses multiple criteria either in addition to or in combination with tests to evaluate entering students.

Questions 3 and 4 on page 4 and questions 7 and 8 on page 6 address evaluation criteria other than tests. Please use those questions to report the other types of evaluation your institution performs.

The test scores used by my institution to determine need for developmental or remedial courses can vary depending on some additional criteria (e.g., high school grades).

Please report the highest score used by your institution that indicates student need for developmental or remedial coursework. You can also use the comment boxes in the questionnaire to provide additional information.

Do transfer students count as “entering students”?

“Entering students” are any students who are new to your institution and are subject to your institution’s policy for determining need for developmental or remedial courses. Please include transfer students if they are eligible for evaluation under your institution’s policy.

My institution does not have a “liberal arts and sciences” program. Or, what do you mean by “liberal arts and sciences”?

The term “liberal arts and sciences” is intended to refer to your institution’s general undergraduate academic degree programs. These include the academic programs in which entering students most frequently enroll when pursuing an undergraduate degree from your institution. We are interested in how your institution evaluates whether entering students who are enrolled in such programs are in need of developmental or remedial courses in mathematics or reading.

5.2 General Questions About Completing the Survey

How do I access the online survey?

The web address of the survey is www.nagb-survey.org. [INTERVIEWER: SPELL OUT ADDRESS IF NEEDED.] You will need to enter the user ID and password shown on the label that was attached to the cover letter we sent. [INTERVIEWER: PROVIDE USER ID AND PASSWORD IF NEEDED.]

Why is this survey important? Or, what is the purpose of this study?

This survey will provide the only source of nationally representative data about the tests and cut-scores used by two-year and four-year institutions to determine student need for remediation in mathematics and reading. The academic preparedness of entering students for entry-level courses is a major concern for many higher education institutions. Your institution’s participation will help address a knowledge gap on this vital issue.

Why was my institution selected?

Your institution was randomly chosen to participate in this study. Your institution’s participation is very important for the success of the study because the answers you provide will be used to represent other institutions similar to yours.

My institution is really not typical of other schools.

That may be true, but we still need your cooperation. Even if your institution is atypical, your information will help to provide comprehensive national estimates.

Do I have to do this?

Your institution’s participation is voluntary, and there is no penalty if you choose not to answer any or all questions in this survey. However, your institution’s participation is very important for the success of the study because the answers you provide will be used to represent other institutions similar to yours.

Who is the sponsor of this survey? Or, what is the National Assessment Governing Board?

The National Assessment Governing Board is the sponsor of this survey. The Governing Board was established by Congress in 1988 to oversee and set policy for the National Assessment of Educational Progress (NAEP), also known as the Nation’s Report Card. NAEP reports regularly to the public on the academic achievement of 4th, 8th, and 12th grade students in reading, writing, mathematics, science, and other subjects, such U.S. history, civics, geography, economics, and the arts. For more information about the Governing Board, go to www.nagb.org. For more information about NAEP, go to http://nces.ed.gov/nationsreportcard.

What is Westat?

Westat is an independent research firm located in Rockville, Maryland. We are conducting this study for the National Assessment Governing Board.

Will responses from my institution be kept confidential?

Data for all participating institutions will be kept confidential. Only aggregate results will be reported; that is, the information provided by your institution will be combined with other participating institutions to produce statistical summaries and reports.

How will the information my institution provides be used?

The National Assessment Governing Board has undertaken a program of research on the academic preparedness of 12th grade students for placement into entry-level credit-bearing college courses. The program of research involves almost 20 studies, of which this survey is one part. The collective results of all of the research studies will be used by the Governing Board to construct and support valid statements about 12th grade academic preparedness in reading and mathematics.

Your institution’s answers will be used only for statistical purposes and will not be disclosed, or used, in identifiable form for any other purpose. Because this is a pilot study designed to improve the questionnaire, your answers will help ensure that the full survey to be conducted later this year includes questions that are clear, relevant, and easily completed. A copy of the results of this study will be sent to all participating institutions.

How much time will it take to complete the survey?

The survey is designed to be completed in 45 minutes or less, including the time for reviewing instructions and completing and reviewing the collection of information.

Will my institution be asked to do this again for the full-scale survey administration?

Some institutions with unique characteristics may be asked to participate in the full-scale survey later this year. This is to ensure that our sample is an accurate representation of the full range of higher education institutions.

Attachments

Attachment 1 Cover Letter to the President/Chancellor of the sampled institution

DATE

Dear President or Chancellor:

On behalf of the Governing Board of the National Assessment of Educational Progress (NAEP), I am asking for your institution’s participation in a pilot test of a survey on the tests used by postsecondary institutions to determine whether students are academically prepared or in need of developmental or remedial courses. This survey is part of a comprehensive research program on the preparedness of 12th grade students for postsecondary education and training.

This study has been endorsed by a number of national higher education organizations that encourage your participation (see overleaf).

The Federal Office of Management and Budget (OMB) has approved this survey (OMB No.: XXXX-XXXX). Westat, a research company in Rockville, Maryland, is contracted to conduct the survey.

Please forward the enclosed survey materials and a copy of this letter to the person or persons most knowledgeable about the evaluation of entering students to determine need for developmental or remedial courses in mathematics and reading. This may include individuals in offices of academic deans or academic departments, or offices that handle student services, new student orientation, or admissions. If more than one office or department is involved, please forward to the office or department with primary responsibility for evaluating entering students to determine need for developmental or remedial courses.

The survey is designed to be completed in 45 minutes or less. Participation in this survey is voluntary. Your institution’s participation is very important for the success of the study because the answers provided will be used to represent other institutions similar to yours. The information provided by your institution will be combined with other participating institutions to produce statistical summaries and reports. Your institution’s answers will be used only for statistical purposes and will not be disclosed, or used, in identifiable form for any other purpose. Answers to frequently asked questions (FAQs) about the study can be found on page 7 of the enclosed survey questionnaire.

Institutions are encouraged to complete the survey online at www.nagb-survey.org. To login, use the username and password printed on the label below.

If it is preferable, the enclosed paper questionnaire may be completed and returned either by mail using the enclosed postage-paid envelope or by toll-free fax at 1-800-254-0984 (Attn: Liam Ristow). The survey should be completed only once, using either the online or paper version.

We ask that the survey be completed within three weeks, and that a copy is kept for your files. If you have any questions about the survey, please contact Liam Ristow, the Westat survey manager, at 1-800-937-8281, ext. 2456 (toll-free) or 240-314-2456, or by e-mail at liamristow@westat.com. Thank you for your assistance.

Sincerely,

Ray Fields

Assistant Director for Policy and Research

Enclosures

800 NORTH CAPITOL STREET, NW, SUITE 825 ‌‌‌ WASHINGTON, DC 20002

The following organizations have endorsed this study and encourage your institution’s participation:

The American Association of Community Colleges

One Dupont Circle NW

Suite 410

Washington, DC 20036

The American Association of State Colleges and Universities

1307 New York Avenue NW

Washington, DC 20005

The American Council on Education

One Dupont Circle NW

Washington, DC 20036

The Association of Land-grant and Public Universities

1307 New York Avenue NW

Suite 400

Washington, DC 20005-4722

The State Higher Education Executive Officers

3035 Center Green Drive
Suite 100
Boulder, CO 80301-2205

Attachment 2 Cover Letter to the Respondent at the sampled institution

DATE

Dear Survey Respondent:

On behalf of the Governing Board of the National Assessment of Educational Progress (NAEP), I am asking for your participation in a pilot test of a survey on the tests used by postsecondary institutions to determine whether students are academically prepared or in need of developmental or remedial courses. This survey is part of a comprehensive research program on the preparedness of 12th grade students for postsecondary education and training. We received your name as the appropriate person to complete the survey through our contact with your institution’s president or chancellor.

This study has been endorsed by a number of national higher education organizations that encourage your institution’s participation (see overleaf).

The Federal Office of Management and Budget (OMB) has approved this survey (OMB No.: XXXX-XXXX). Westat, a research company in Rockville, Maryland, is contracted to conduct the survey.

You were identified as the person most knowledgeable about your institution’s policy on evaluating entering students to determine need for developmental or remedial courses in mathematics and reading. However, you may need to seek out some information from other locations within your institution to complete the survey. We greatly appreciate your efforts to collaborate with other offices or personnel to gather the information requested on the survey.

The survey is designed to be completed in 45 minutes or less. Participation in this survey is voluntary. Your participation is very important for the success of the study because the answers provided will be used to represent other institutions similar to yours. The information you provide will be combined with other participating institutions to produce statistical summaries and reports. Your answers will be used only for statistical purposes and will not be disclosed, or used, in identifiable form for any other purpose. Answers to frequently asked questions (FAQs) about the study can be found on page 7 of the enclosed survey questionnaire.

We encourage you to complete the survey online at www.nagb-survey.org. To login, enter the username and password shown below.

If you prefer, the paper questionnaire may be completed and returned either by mail using the address shown on the cover page of the questionnaire or by toll-free fax at 1-800-254-0984 (Attn: Liam Ristow). The survey should be completed only once, using either the online or paper version.

We ask that you complete the survey within three weeks, and that a copy is kept for your files. If you have any questions about the survey, please contact Liam Ristow, the Westat survey manager, at 1-800-937-8281, ext. 2456 (toll-free) or 240-314-2456, or by e-mail at liamristow@westat.com. Thank you for your assistance.

Sincerely,

Ray Fields

Assistant Director for Policy and Research

Enclosures

800 NORTH CAPITOL STREET, NW, SUITE 825 ‌‌‌ WASHINGTON, DC 20002

The following organizations have endorsed this study and encourage your institution’s participation:

The American Association of Community Colleges

One Dupont Circle NW

Suite 410

Washington, DC 20036

The American Association of State Colleges and Universities

1307 New York Avenue NW

Washington, DC 20005

The American Council on Education

One Dupont Circle NW

Washington, DC 20036

The Association of Land-grant and Public Universities

1307 New York Avenue NW

Suite 400

Washington, DC 20005-4722

The State Higher Education Executive Officers

3035 Center Green Drive
Suite 100
Boulder, CO 80301-2205

Attachment 3 Survey Questionnaire for 2-year institutions

National Assessment Governing Board

WASHINGTON, D.C. 20002

EVALUATING STUDENT NEED FOR DEVELOPMENTAL OR REMEDIAL

COURSES AT POSTSECONDARY EDUCATION INSTITUTIONS

FORM APPROVED

O.M.B. No.:

EXPIRATION DATE:

Please respond for the institution that matches the IPEDS ID number printed on the label below. If the information for the institution shown is incorrect, please update directly on the label before returning the survey questionnaire.

OMB DRAFT 6-17-10

VERSION FOR TWO-YEAR INSTITUTIONS

LABEL WITH IPEDS ID HERE

This survey can be completed online at www.nagb-survey.org. We encourage you to complete the survey online if possible. You will need the User ID and Password listed on the attached cover letter to log in to the website. Please contact Liam Ristow at liamristow@westat.com, 1-800-937-8281, Ext. 2456 (toll free) or 240-314-2456 if you do not have your User ID or Password.

If you prefer, you may complete this paper version. If you complete the paper version, please provide the following information, keep a copy of the completed questionnaire for your files, and return the original to Westat at the address shown below. We have enclosed a postage-paid envelope for your convenience. You may also fax a copy of the completed questionnaire to 1-800-254-0984 (Attn: Liam Ristow).

Name:

Title/position:

Telephone number: E-mail:

Indicate the number of minutes it took to complete this questionnaire in the space below. Include time to review instructions, search existing records, and gather the information needed to complete the questionnaire.

Number of minutes: _______________

Thank you. Please keep a copy of the survey for your records.

Please see page 7 for a list of Frequently Asked Questions (FAQs) regarding the purpose of this study.

PLEASE RETURN COMPLETED FORM TO: IF YOU HAVE ANY QUESTIONS OR COMMENTS, CONTACT:

Mail: Liam Ristow (8756.05.03) Liam Ristow at Westat

Westat 1-800-937-8281, Ext. 2456 or 240-314-2456

1600 Research Boulevard E-mail: liamristow@westat.com
Rockville, Maryland 20850-3195

Fax: 800-254-0984

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is xxxx-xxxx. The time required to complete this information collection is estimated to average 45 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate or suggestions for improving this form, OR if you have any comments or concerns regarding the status of your individual submission of this form, please write directly to National Assessment Governing Board, 800 North Capitol Street, NW, Washington, DC 20002.

Attachment 3 Survey Questionnaire for 2-year institutions (continued)

urvey Overview and Instructions

Admissions tests such as the ACT or SAT;
Placement tests such as ACCUPLACER, ASSET, and COMPASS; or
Institution- or state-developed tests.

Please report based on the tests your institution uses to evaluate entering students who are pursuing a degree program that is designed to transfer to a four-year institution.

Definitions

Please note:

In most cases, developmental or remedial courses are not credit-bearing and they do not count toward general education or degree requirements.
Your institution may use other names such as “preparatory,” "compensatory," or "basic skills" or some other term to refer to developmental or remedial courses.

Attachment 3 Survey Questionnaire for 2-year institutions (continued)

irections: If your institution did not have any entering students in fall 2009 who were pursuing a degree program designed to transfer to a four-year institution, please check this box , complete the cover page, and return the questionnaire.

Section A. Evaluating Need for Developmental or Remedial Mathematics Courses

1. In fall 2009, did your institution use ACT, SAT, or placement tests (i.e., ACCUPLACER, ASSET, COMPASS, or other tests developed by your institution or state) to evaluate whether entering students were in need of developmental or remedial mathematics courses (i.e., not academically prepared for entry-level courses that require college mathematics skills)?

Consider tests used to evaluate entering students who were pursuing a degree program designed to transfer to a four-year institution.
Consider any use of ACT, SAT, or placement test scores to determine the need for remediation, even if your institution does not offer developmental or remedial mathematics courses or offers only one or two such courses.

Yes 1 (Continue with question 2.)

No 2 (Skip to question 3.)

2. In Column B, please check the box for each ACT, SAT, or placement test that your institution used in fall 2009 to evaluate whether entering students were in need of developmental or remedial mathematics courses. In Column C, for each test used, write the score below which students were identified as in need of developmental or remedial mathematics courses.

If different scores were used for either requiring or recommending developmental or remedial mathematics courses, report the highest score used.

If different scores were used to identify students for different levels of developmental or remedial mathematics courses, report the score used for the highest level of remedial mathematics course.

A. ACT, SAT, or placement test (Score ranges shown in parentheses)	B. Test used to evaluate entering students	C. Score below which developmental or remedial mathematics courses were needed
ACT Subject Tests
a. Mathematics (1-36)		________
b. Science (1-36)		________
c. Composite score (1-36)		________
SAT Reasoning Test
d. Mathematics (200-800)		________
e. SAT total score including Writing (600-2400)		________
f. SAT total score excluding Writing (400-1600)		________
ACCUPLACER
g. Arithmetic (20-120)		________
h. Elementary Algebra (20-120)		________
i. College-Level Mathematics (20-120)		________
ASSET
j. Numerical Skills (23-55)		________
k. Elementary Algebra (23-55)		________
l. Intermediate Algebra (23-55)		________
m. College Algebra (23-55)		________
n. Geometry (23-55)		________
COMPASS
o. Pre-Algebra placement domain (1-99)		________
p. Algebra placement domain (1-99)		________
q. College Algebra placement domain (1-99)		________
r. Geometry placement domain (1-99)		________
s. Trigonometry placement domain (1-99)		________
Other mathematics placement tests
t. Other test 1 (specify):		Not applicable.
u. Other test 2 (specify):
v. Other test 3 (specify):

Attachment 3 Survey Questionnaire for 2-year institutions (continued)

lease provide any comments about your response to question 2.

3. In fall 2009, did your institution use any criteria other than ACT/SAT tests or placement tests to evaluate whether entering students were in need of developmental or remedial mathematics courses (i.e., not academically prepared for entry-level courses that require college mathematics skills)?

Consider other criteria such as high school graduation or end-of-course exams, high school records, Advanced Placement (AP) or International Baccalaureate (IB) test scores, or faculty recommendations.

Yes 1 (Continue with question 4.)

No 2 (Skip to question 5.)

4. Please check the box for each criterion that your institution used in fall 2009 to evaluate whether entering students were in need of developmental or remedial mathematics courses (i.e., not academically prepared for entry-level courses that require college mathematics skills).

Criterion	Used to evaluate entering students
a. High school graduation tests or end-of-course tests
b. High school grades (including grade point average)
c. Highest high school mathematics course completed
d. Advanced Placement (AP) or International Baccalaureate (IB) test scores
e. Faculty recommendation
f. Other (specify)

Please provide any comments about your response to question 4.

Section B. Evaluating Need for Developmental or Remedial Reading Courses

Please note: Developmental or remedial writing courses should not be considered reading courses unless they have a substantial focus on improving reading skills.

5. In fall 2009, did your institution use ACT, SAT, or placement tests (i.e., ACCUPLACER, ASSET, COMPASS, or other tests developed by your institution or state) to evaluate whether entering students were in need of developmental or remedial reading courses (i.e., not academically prepared for entry-level courses that require college reading skills)?

Consider tests used to evaluate entering students who were pursuing a degree program designed to transfer to a four-year institution.
Consider any use of ACT, SAT, or placement test scores to determine the need for remediation, even if your institution does not offer developmental or remedial reading courses or offers only one or two such courses.

Yes 1 (Continue with question 6.)

No 2 (Skip to question 7.)

6. In Column B, please check the box for each ACT, SAT or placement test that your institution used in fall 2009 to evaluate whether entering students were in need of developmental or remedial reading courses. In Column C, for each test used, write the score below which students were identified as in need of developmental or remedial reading courses.

I
Attachment 3 Survey Questionnaire for 2-year institutions (continued)
f different scores were used for either requiring or recommending developmental or remedial reading courses, report the highest score used.

If different scores were used to identify students for different levels of developmental or remedial reading courses, report the score used for the highest level of remedial reading course.

A. ACT, SAT, or placement test (Score ranges shown in parentheses)	B. Test used to evaluate entering students	C. Score below which developmental or remedial reading courses were needed
ACT Subject Tests
a. Reading (1-36)		________
b. English (1-36)		________
c. Writing (1-36)		________
d. Composite score (1-36)		________
SAT Reasoning Test
e. Critical Reading (200-800)		________
f. Writing (200-800)		________
g. SAT total score including Writing (600-2400)		________
h. SAT total score excluding Writing (400-1600)		________
ACCUPLACER
i. Reading Comprehension (20-120)		________
j. Sentence Skills (20-120)		________
k. WritePlacer (1-8)		________
ASSET
l. Reading Skills (23-55)		________
m. Writing Skills (23-55)		________
COMPASS
n. Reading placement domain (1-99)		________
o. Writing Skills placement domain (1-99)		________
p. Writing e-Write placement domain (2-8)		________
q. Writing e-Write placement domain (2-12)		________
Nelson-Denny Reading Test r. Nelson-Denny Reading test (0-172)		________
Other reading placement tests
s. Other test 1 (specify):		Not applicable.
t. Other test 2 (specify):
u. Other test 3 (specify):

Please provide any comments about your response to question 6.

7. In fall 2009, did your institution use any criteria other than ACT/SAT tests or placement tests to evaluate whether entering students were in need of developmental or remedial reading courses (i.e., not academically prepared for entry-level courses that require college reading skills)?

Consider other criteria such as high school graduation or end-of-course exams, high school records, Advanced Placement (AP) or International Baccalaureate (IB) test scores, or faculty recommendations.

Yes 1 (Continue with question 8.)

No 2 (Stop. Complete cover page and return questionnaire.)

Attachment 3 Survey Questionnaire for 2-year institutions (continued)

. Please check the box for each criterion that your institution used in fall 2009 to evaluate whether entering students were in need of developmental or remedial reading courses (i.e., not academically prepared for entry-level courses that require college reading skills).

Criterion	Used to evaluate entering students
a. High school graduation tests or end-of-course tests
b. High school grades (including grade point average)
c. Highest high school English course completed
d. Advanced Placement (AP) or International Baccalaureate (IB) test scores
e. Faculty recommendation
f. Other (specify)

Please provide any comments about your response to question 8.

Thank you for completing the survey. Please remember to complete the information on the cover page (name, contact information, and time needed to complete the survey) before returning the questionnaire.

FREQUENTLY ASKED QUESTIONS

Attachment 3 Survey Questionnaire for 2-year institutions (continued)

Why is this survey important?

The academic preparedness of entering students for entry-level courses is a major concern for many higher education institutions. This survey will provide the only source of nationally representative data about the tests and cut-scores used by two-year and four-year institutions to determine student need for remediation in mathematics and reading. Thus, your institution’s participation will help address a knowledge gap on this vital issue.

Why was my institution selected? Do I have to do this?

Your institution was randomly chosen to participate in this study. Your institution’s participation is voluntary, and there is no penalty if you choose not to answer any or all questions in this survey. Your institution’s participation is very important for the success of the study because the answers you provide will be used to represent other institutions similar to yours.

Who is the sponsor of this survey?

The National Assessment Governing Board is the sponsor of this survey. The Governing Board was established by Congress in 1988 to oversee and set policy for the National Assessment of Educational Progress (NAEP), also known as the Nation’s Report Card. NAEP reports regularly to the public on the academic achievement of 4th, 8th and 12th grade students in reading, writing, mathematics, science, and other subjects, such U.S. history, civics, geography, economics, and the arts. For more information about NAEP, go to http://nces.ed.gov/nationsreportcard.

Will responses from my institution be kept confidential?

How will the information my institution provides be used?

Your institution’s answers will be used only for statistical purposes and will not be disclosed, or used, in identifiable form for any other purpose. A copy of the results of this study will be sent to all participating institutions.

The National Assessment Governing Board has undertaken a program of research to enable NAEP to report on the academic preparedness of 12th grade students for placement into entry-level credit-bearing college courses. The program of research involves almost 20 studies, of which this survey is one part. The collective results of all of the research studies will be used by the Governing Board to construct and support valid statements about 12th grade academic preparedness in reading and mathematics.

Who is conducting the survey?

Westat is conducting this survey under contract to the National Assessment Governing Board. Westat is a research company located in Rockville, Maryland.

How much time will it take to complete the survey?

The survey is designed to be completed in 45 minutes or less, including the time for reviewing instructions and completing and reviewing the collection of information.

Whom do I contact if I have questions?

For questions about the survey instructions or survey items, please contact Liam Ristow, the Westat survey manager, at 1-800-937-8281, ext. 2456 (toll-free) or 240-314-2456, or by e-mail at liamristow@westat.com.

For questions about the National Assessment Governing Board and its research program, please contact Ray Fields, Assistant Director for Policy and Research, National Assessment Governing Board, at 202-357-0395, or by e-mail at Ray.Fields@ed.gov.

Attachment 4 Survey Questionnaire for 4-year institutions

National Assessment Governing Board

WASHINGTON, D.C. 20002

EVALUATING STUDENT NEED FOR DEVELOPMENTAL OR REMEDIAL

COURSES AT POSTSECONDARY EDUCATION INSTITUTIONS

FORM APPROVED

O.M.B. No.:

EXPIRATION DATE:

OMB DRAFT 6-17-10

VERSION FOR FOUR-YEAR INSTITUTIONS

LABEL WITH IPEDS ID HERE

Name:

Title/position:

Telephone number: E-mail:

Number of minutes: _______________

Thank you. Please keep a copy of the survey for your records.

Please see page 7 for a list of Frequently Asked Questions (FAQs) regarding the purpose of this study.

PLEASE RETURN COMPLETED FORM TO: IF YOU HAVE ANY QUESTIONS OR COMMENTS, CONTACT:

Mail: Liam Ristow (8756.05.03) Liam Ristow at Westat

Westat 1-800-937-8281, Ext. 2456 or 240-314-2456

1600 Research Boulevard E-mail: liamristow@westat.com
Rockville, Maryland 20850-3195

Fax: 1-800-254-0984

Attachment 4 Survey Questionnaire for 4-year institutions (continued)

urvey Overview and Instructions

Admissions tests such as the ACT or SAT;
Placement tests such as ACCUPLACER, ASSET, and COMPASS; or
Institution- or state-developed tests.

Please report based on the tests your institution uses to evaluate entering students who are enrolled in an undergraduate degree program in the liberal arts and sciences.

Definitions

Please note:

In most cases, developmental or remedial courses are not credit-bearing and they do not count toward general education or degree requirements.
Your institution may use other names such as “preparatory,” "compensatory," or "basic skills" or some other term to refer to developmental or remedial courses.

Attachment 4 Survey Questionnaire for 4-year institutions (continued)

irections: If your institution did not have any entering students in fall 2009 who were enrolled in an undergraduate degree program in the liberal arts and sciences, please check this box , complete the cover page, and return the questionnaire.

Section A. Evaluating Need for Developmental or Remedial Mathematics Courses

Consider tests used to evaluate entering students who were enrolled in an undergraduate degree program in the liberal arts and sciences.
Consider any use of ACT, SAT, or placement test scores to determine the need for remediation, even if your institution does not offer developmental or remedial mathematics courses or offers only one or two such courses.

Yes 1 (Continue with question 2.)

No 2 (Skip to question 3.)

If different scores were used for either requiring or recommending developmental or remedial mathematics courses, report the highest score used.

If different scores were used to identify students for different levels of developmental or remedial mathematics courses, report the score used for the highest level of remedial mathematics course.

A. ACT, SAT, or placement test (Score ranges shown in parentheses)	B. Test used to evaluate entering students	C. Score below which developmental or remedial mathematics courses were needed
ACT Subject Tests
a. Mathematics (1-36)		________
b. Science (1-36)		________
c. Composite score (1-36)		________
SAT Reasoning Test
d. Mathematics (200-800)		________
e. SAT total score including Writing (600-2400)		________
f. SAT total score excluding Writing (400-1600)		________
ACCUPLACER
g. Arithmetic (20-120)		________
h. Elementary Algebra (20-120)		________
i. College-Level Mathematics (20-120)		________
ASSET
j. Numerical Skills (23-55)		________
k. Elementary Algebra (23-55)		________
l. Intermediate Algebra (23-55)		________
m. College Algebra (23-55)		________
n. Geometry (23-55)		________
COMPASS
o. Pre-Algebra placement domain (1-99)		________
p. Algebra placement domain (1-99)		________
q. College Algebra placement domain (1-99)		________
r. Geometry placement domain (1-99)		________
s. Trigonometry placement domain (1-99)		________
Other mathematics placement tests
t. Other test 1 (specify):		Not applicable.
u. Other test 2 (specify):
v. Other test 3 (specify):

Attachment 4 Survey Questionnaire for 4-year institutions (continued)

Please provide any comments about your response to question 2.

Consider other criteria such as high school graduation or end-of-course exams, high school records, Advanced Placement (AP) or International Baccalaureate (IB) test scores, or faculty recommendations.

Yes 1 (Continue with question 4.)

No 2 (Skip to question 5.)

Criterion	Used to evaluate entering students
a. High school graduation tests or end-of-course tests
b. High school grades (including grade point average)
c. Highest high school mathematics course completed
d. Advanced Placement (AP) or International Baccalaureate (IB) test scores
e. Faculty recommendation
f. Other (specify)

Please provide any comments about your response to question 4.

Section B. Evaluating Need for Developmental or Remedial Reading Courses

Please note: Developmental or remedial writing courses should not be considered reading courses unless they have a substantial focus on improving reading skills.

Consider tests used to evaluate entering students who were enrolled in an undergraduate degree program in the liberal arts and sciences.
Consider any use of ACT, SAT, or placement test scores to determine the need for remediation, even if your institution does not offer developmental or remedial reading courses or offers only one or two such courses.

Yes 1 (Continue with question 6.)

No 2 (Skip to question 7.)

6. In Column B, please check the box for each ACT, SAT, or placement test that your institution used in fall 2009 to evaluate whether entering students were in need of developmental or remedial reading courses. In Column C, for each test used, write the score below which students were identified as in need of developmental or remedial reading courses.

I
Attachment 4 Survey Questionnaire for 4-year institutions (continued)
f different scores were used for either requiring or recommending developmental or remedial reading courses, report the highest score used.

If different scores were used to identify students for different levels of developmental or remedial reading courses, report the score used for the highest level of remedial reading course.

A. ACT, SAT, or placement test (Score ranges shown in parentheses)	B. Test used to evaluate entering students	C. Score below which developmental or remedial reading courses were needed
ACT Subject Tests
a. Reading (1-36)		________
b. English (1-36)		________
c. Writing (1-36)		________
d. Composite score (1-36)		________
SAT Reasoning Test
e. Critical Reading (200-800)		________
f. Writing (200-800)		________
g. SAT total score including Writing (600-2400)		________
h. SAT total score excluding Writing (400-1600)		________
ACCUPLACER
i. Reading Comprehension (20-120)		________
j. Sentence Skills (20-120)		________
k. WritePlacer (1-8)		________
ASSET
l. Reading Skills (23-55)		________
m. Writing Skills (23-55)		________
COMPASS
n. Reading placement domain (1-99)		________
o. Writing Skills placement domain (1-99)		________
p. Writing e-Write placement domain (2-8)		________
q. Writing e-Write placement domain (2-12)		________
Nelson-Denny Reading Test r. Nelson-Denny Reading test (0-172)		________
Other reading placement tests
s. Other test 1 (specify):		Not applicable.
t. Other test 2 (specify):
u. Other test 3 (specify):

Please provide any comments about your response to question 6.

Consider other criteria such as high school graduation or end-of-course exams, high school records, Advanced Placement (AP) or International Baccalaureate (IB) test scores, or faculty recommendations.

Yes 1 (Continue with question 8.)

No 2 (Stop. Complete cover page and return questionnaire.)

Attachment 4 Survey Questionnaire for 4-year institutions (continued)

8. Please check the box for each criterion that your institution used in fall 2009 to evaluate whether entering students were in need of developmental or remedial reading courses (i.e., not academically prepared for entry-level courses that require college reading skills).

Criterion	Used to evaluate entering students
a. High school graduation tests or end-of-course tests
b. High school grades (including grade point average)
c. Highest high school English course completed
d. Advanced Placement (AP) or International Baccalaureate (IB) test scores
e. Faculty recommendation
f. Other (specify)

Please provide any comments about your response to question 8.

Thank you for completing the survey. Please remember to fill in the information on the cover page (name, contact information, and time needed to complete the survey) before returning the questionnaire.

FREQUENTLY ASKED QUESTIONS

Attachment 4 Survey Questionnaire for 4-year institutions (continued)

Why is this survey important?

Why was my institution selected? Do I have to do this?

Who is the sponsor of this survey?

The National Assessment Governing Board is the sponsor of this survey. The Governing Board was established by Congress in 1988 to oversee and set policy for the National Assessment of Educational Progress (NAEP), also known as the Nation’s Report Card. NAEP reports regularly to the public on the academic achievement of 4th, 8th and 12th grade students in reading, writing, mathematics, science, and other subjects, such U.S. history, civics, geography, economics, and the arts. For more information about NAEP, go to http://nces.ed.gov/nationsreportcard.

Will responses from my institution be kept confidential?

How will the information my institution provides be used?

Who is conducting the survey?

Westat is conducting this survey under contract to the National Assessment Governing Board. Westat is a research company located in Rockville, Maryland.

How much time will it take to complete the survey?

The survey is designed to be completed in 45 minutes or less, including the time for reviewing instructions and completing and reviewing the collection of information.

Whom do I contact if I have questions?

Attachment 5 Remail Request Form

Remail Request Form

NAGB

Evaluating Student Need for Development or Remedial Courses at Postsecondary Education Institutions

Project Number: 8756.11.01

Date: Interviewer:

Unit ID (2-year): Unit ID (4-year):

This remail is to (check one): President’s office  Designated Respondent 

Name:

Title:

Office/Dept.:

Institution Name:

Building/Room No.:

Street Address:

City, State, Zip:

Telephone No.:

Comments:

Date mailed:

Attachment 6 Fax Request Form

Fax Request Form

NAGB

Evaluating Student Need for Development or Remedial Courses at Postsecondary Education Institutions

Project Number: 8756.11.01

Date: Interviewer:

Unit ID (2-year): Unit ID (4-year):

This fax is to (check one): President’s office  Designated Respondent 

Name:

Title:

Office/Dept.:

Institution Name:

Fax No.:

Telephone No.:

Comments:

Date faxed:

Attachment 6a Fax Cover Sheet

Fax

Cover sheet plus (Page Num) page(s)

Date
To
Institution
Fax
From
Phone	1-800-937-8288 ext.
Fax	1-800-254-0984 (toll-free)

Comments

This message is intended for the sole use of the individual or entity to which it is addressed and may contain information that is proprietary, confidential, and protected from disclosure under applicable law. If you are not the intended recipient or are not responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this fax in error, please notify Westat Security Services immediately by telephone at 1-866-937-7327 (toll-free) and return the original message to us at the above address by return mail. We will reimburse any reasonable costs you incur in notifying us and returning the fax to us. Thank you.

If you do not receive all pages or transmission is unclear, please notify sender identified above.

Attachment 7 Email Request Form

Email Request Form

NAGB

Evaluating Student Need for Development or Remedial Courses at Postsecondary Education Institutions

Project Number: 8756.11.01

Date: Interviewer:

Unit ID (2-year): Unit ID (4-year):

This email is to (check one): President’s office  Designated Respondent 

Name:

Title:

Office/Dept.:

Institution Name:

Telephone No.:

Email address:

Comments:

Date email sent:

Attachment 7a Email Cover Sheet

E-mail

Date
To
Institution
E-mail
From
Phone	1-800-937-8288 ext.
Fax	1-800-254-0984 (toll-free)

Comments

If you do not receive all pages or transmission is unclear, please notify sender identified above.

Attachment 8 Non-Interview Report Form

NON-INTERVIEW REPORT FORM (NIRF)

NAGB

Evaluating Student Need for Development or Remedial Courses at Postsecondary Education Institutions

Project Number: 8756.11.01

Unit ID (2-year): Unit ID (4-year):

Date:

Interviewer:

1. REFUSAL/BREAKOFF MADE BY:

President’s office contact…………………………………………………………….1

Designated respondent……………………………………………………………….2

Other………………………………………………………………………………… 3

2. STATE REASON FOR REFUSAL/BREAKOFF: (RECORD VERBATIM)

3. CIRCLE ALL REASONS FOR REFUSAL/BREAKOFF:

a. Information not available 1

b. Too busy to complete list 2

c. Negative reaction to surveys in general 3

d. Survey sounds too long 4

e. No reason given 5

f. Other 6

STRENGTH OF REFUSAL/BREAKOFF:

Mild, no hostility 1

Firm, but not hostile 2

Hostile 3

5. RECOMMENDED ACTION:

Attachment 9 Interviewer Problem Sheet

INTERVIEWER PROBLEM SHEET

NAGB

ID #___________ DATE:___________ INITIALS ______________

PROBLEM

QUESTION #	EXPLANATION









RESOLUTION
QUESTION #	EXPLANATION

Attachment 10 Respondent Information Sheet (RIS)

Evaluating Student Need for Development or Remedial Courses at Postsecondary Education Institutions

National Assessment Governing Board (NAGB) Pilot Study

August 2010

Please document how easy or difficult it was for the President/Chancellor’s office to distribute our request to an appropriate office.

Did they know exactly whom the survey should have been directed to?

Once you reached the designated respondent were they able and willing to coordinate completion of the survey from other departments? For e.g., did the Math department know whom to contact in the English department to get information for our survey? Or vice-versa? Please give us as much information as possible.

COMMENTS:

Evaluating Student Need for Developmental or Remedial Courses at Postsecondary Education Institutions

I. PRESIDENT/CHANCELLOR’S OFFICE CONTACT

CONTACT INSTITUTION AND ASK TO BE CONNECTED TO THE OFFICE OF THE PRESIDENT OR CHANCELLOR. WHEN YOU HAVE REACHED THE OFFICE OF THE PRESIDENT OR CHANCELLOR, READ SCRIPT BELOW.

Hello, my name is (YOUR NAME). I'm calling from Westat on behalf of the Governing Board of the National Assessment of Education Progress (NAEP). Recently we sent your office an ivory-colored/yellow-colored questionnaire about the preparedness of 12^th grade students for postsecondary education and training. This pilot study is designed to obtain information about tests and test scores used to place students into developmental or remedial coursework.

Did you receive the questionnaire? (EXPLAIN CONTENTS OF PACKAGE)

(1) Cover letter addressed to President/Chancellor, and (2) an ivory-colored/yellow-colored questionnaire

Yes (READ PARAGRAPH A)

No (READ PARAGRAPH B)

A. PACKAGE RECEIVED

We are calling to check on the status of the package. The package included a letter requesting that the survey and accompanying materials be sent to the staff member/office that is most knowledgeable about the evaluation of entering students to determine need for developmental or remedial courses in mathematics and reading. To which office would this questionnaire have been directed?

(PROMPT IF NEEDED: Because the survey asks about tests and test scores used to place students into developmental or remedial coursework, many universities refer the survey to the student services, placement office, math department, or admissions office. See list of offices on Page 3-3 in Chapter 3 of your manual.)

1. May I have the name and contact information of the person/office to whom you have directed this questionnaire?

Yes (RECORD NAME AND CONTACT INFORMATION IN PART II, DESIGNATED RESPONDENT INFORMATION; CALL DESIGNATED RESPONDENT)

Yes. There are multiple potential respondents. To whom would you suggest we send your request? (INTERVIEWER, PLEASE GO TO PAGE 3-3 IN CHAPTER 3 OF THE MANUAL AND SUGGEST THE LOCATIONS OF POSSIBLE APPROPRIATE RESPONDENTS.

RECORD NAME AND CONTACT INFORMATION IN PART II, DESIGNATED RESPONDENT INFORMATION; CALL DESIGNATED RESPONDENT)

Not yet sent/President has not reviewed request/President has not appointed respondent

Would it be possible for you to check on the status of our request? This is a pilot test being conducted in a short period of time. I will call back to check which office/respondent the request was sent. When would be a convenient time for me to call back?

CALLBACK DATE:___________________________________________________

Thank you for your time.

B. SURVEY NOT RECEIVED

This pilot test is being conducted for the Governing Board of the National Assessment of Education Progress (NAEP) and is designed to obtain information about tests and test scores used to place students into developmental or remedial coursework.

It is designed to be completed by the person or office at your institution that is most knowledgeable about these placement tests. Many institutions forward the questionnaire to the student services, placement office, the math department, or admissions office.

(INTERVIEWER, PLEASE GO TO PAGE 3-3 IN CHAPTER 3 OF THE MANUAL AND SUGGEST THE LOCATIONS OF POSSIBLE APPROPRIATE RESPONDENTS.)

Would it be possible to fax the survey directly to the appropriate person or office?

NO, send to President/Chancellor’s office

Since we are trying to complete the pilot study within 6 weeks, we would like to fax or email the materials to you? May I please have your fax number/email address?

RECORD FAX NUMBER:_________________________________________

RECORD EMAIL ADDRESS:_________________________________________

I will email/fax the survey request to you and callback to check who the designated respondent will be. Thank you for your time.

YES, send to designated respondent

(RECORD RESPONDENT NAME, ADDRESS, PHONE AND FAX NUMBERS, AND EMAIL ADDRESS IN PART II PAGE 3, DESIGNATED RESPONDENT INFORMATION.

CONTACT DESIGNATED RESPONDENT BEFORE YOU SEND THE EMAIL/FAX TO THEM)

II. RESPONDENT CONTACT - DESIGNATED RESPONDENT

NAME:

TITLE:

OFFICE/DEPARTMENT:

ADDRESS:

TELEPHONE NUMBER:

FAX NUMBER:

EMAIL ADDRESS:

CALL THE NUMBER ABOVE AND ASK TO SPEAK TO THE RESPONDENT. IF RESPONDENT IS NOT AVAILABLE, ASK IF THE STATUS OF THE QUESTIONNAIRE IS KNOWN. AS NEEDED, OBTAIN A CONVENIENT TIME TO CALL BACK.

Hello, my name is (YOUR NAME). I'm calling from Westat on behalf of the Governing Board of the National Assessment of Education Progress (NAEP) about the preparedness of 12^th grade students for postsecondary education and training. This pilot test is designed to obtain information about tests and test scores used to place students into developmental or remedial coursework.

Your President’s office (GIVE NAME) indicated that (HE/SHE) forwarded the package containing the ivory-colored/yellow-colored questionnaire to you. Did you receive the questionnaire?

Yes (READ PARAGRAPH A)

No (READ PARAGRAPH B)

A. SURVEY RECEIVED

We are calling to check on the status of the questionnaire.

Questionnaire completed/submitted

DATE MAILED/SUBMITTED: __________________________________________________

Thank you very much for participating in the survey. We will look for your questionnaire. If we do not receive it soon, we will call back to let you know. Thank you for your time.

Still working on questionnaire

Since this is a pilot test being conducted in a short period of time, we would like to have all questionnaires completed and returned as soon as possible. Can you give us an idea of when we can expect your questionnaire?

EXPECTED DATE OF COMPLETION: ________________________________________________

Please remember that the survey is designed to be completed online. Do you have the web address and password? (DIRECT THE RESPONDENT TO THE PINK OR GREEN LABEL ON THE FRONT OF THE LETTER OR PROVIDE URL, ID AND PASSWORD: www.nagb-survey.org).

Please keep a copy of the completed questionnaire for your records. We will look for your questionnaire; if we do not receive it, we will call back to let you know. What is the best time to reach you?

BEST TIME:____________________________________________

Thank you for your time.

INTERVIEWER: Please direct the respondent to the website and as needed, provide him/her with the URL, User ID, and Password. Let them know that they can print a paper copy of the survey from the website as well. Remind them to keep a copy of the completed questionnaire for their records.

B. SURVEY NOT RECEIVED

This pilot test is being conducted for the National Assessment Governing Board and is designed to obtain information about the preparedness of 12^th grade students for postsecondary education and training.

Since the time-frame to complete this pilot test is short (ONLY 6 WEEKS), I would like to give you the website, your ID and password to log on. You can also print a paper copy of the survey from the website.

We would like to have all questionnaires completed and returned as soon as possible.

Can you give us an idea of when we can expect your questionnaire?

EXPECTED DATE OF COMPLETION:_____________________________________________

If we do not receive the questionnaire, we will call you back. Please remember to keep a copy of the completed questionnaire for your records. Thank you for your time.

Attachment 11 Call Record CALL RECORD

FILE KEY:		TIME ZONE”
PREVIOUS DISPOSITION:		TELEPHONE:
TOTAL CALLS:		APP DATE/TIME:

INTERVIEWER INITIALS		DATE	TIME BEGUN	TIME ENDED		RESULTS	COMMENTS		CALL BACK INFO. DATE TIME	D/E/W









	(1) RING NO ANSWER				(PC) PARTIAL COMPLETE			(C1) COMPLETE 1
	(2) FIRST REFUSAL/BREAKOFF				(OS) OUT OF SCOPE			(C2) COMPLETE 2
	(3) BUSY				(I) INELIGIBLE (MC) MAXIMUM CONTACT			(C3) COMPLETE 3
	(4) CALLBACK - NO APPT.				(OA) OUT OF AREA			(S3) SPECIFIC 3
	(5) CALLBACK - APPT.				(RB) FINAL REFUSAL/BREAKOFF			(S4) SPECIFIC 4
	(6) INITIAL LANGUAGE PROBLEM				(LP) FINAL LANGUAGE PROBLEM			(SR) SPECIFIC REFUSAL CODE
	(7) RETURNED BY MAIL				(O) OTHER			(N1) B.O. CHECK (Residential)
	(8) PROBLEM (Specify)				(NR) NONRESIDENTIAL			(N2) B.O. CHECK (Nonresidential)
	(9) MAILOUT NEEDED				(NA) NO ANSWER			(N3) B.O. CHECK (Working only)
	(10) TRACING NEEDED (11) 800 MESSAGE LEFT				(NW) NON WORKING			(N4) B.O. CHECK (Underdetermined)
	(12) FAX NEEDED				(NL) NON LOCATABLE
	(13) EMAIL NEEDED
	(CP) COMPLETE BY PHONE (CF) COMPLETE BY FAX				(S1) SPECIFIC 1
	(CW) COMPLETE BY WEB				(S2) SPECIFIC 2
	(CM) COMPLETE BY MAIL

CASE ID

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	POSTSECONDARY EDUCATION QUICK INFORMATION SYSTEM (PEQIS)
Author	Nicole L. Bartfai
File Modified	0000-00-00
File Created	2021-02-01

Pilot - Interviewer Training Manual

Pilot - Appendix A interviewer training manual.docx

Evaluating Student Need for Developmental or Remedial Courses at Postsecondary Education Institutions (Formerly titled Survey of Placement Tests and Cut-Scores in Higher Education Institutions).