NRS Instrument Revisions Discussion

Discussion of NRS Instrument Revisions

The NRS instrument consists of 11 required tables and 6 optional tables.

The number of tables will not change.

1. Replace goal-setting with automatic cohort identification as a means for defining the population of students on which States report the NRS follow-up measures of (a) employment, (b) obtaining a secondary credential and (c) entry into postsecondary education or training.

1. Employment Measures

The NRS requires States to identify who among their unemployed students has a goal of “entered employment” and to follow up only with these students after they exit the adult education program to determine whether they have entered employment. States must also follow students who obtain employment after program exit to determine whether they have retained employment.

The data over the last three program years (PYs) indicate that States are under-setting the employment goals. The national percentage of unemployed students who exited the program with employment goals currently ranges from 18 percent to 22 percent. Many States have reported substantially lower percentages of students with employment goals. For example, in PY 2008–2009, one State reported that less than 4 percent of its unemployed students who exited the program had a goal of entering employment.

Furthermore, the number of students with employment-related goals set and met over the last several years fluctuates among States. This variation suggests instability in the data that likely is due to problematic goal setting, such as not setting an employment goal for unemployed students who are seeking a job) and data collection practices rather than differences in the student population over the years. Consequently, it appears that the policy that allows local programs to set individual student goals related to employment results in outcome data that is of limited use as a measure of accountability.

To correct this problem and thereby improve the quality of data, OVAE will discontinue the use of goal-setting as a means for States to identify students to track to determine employment. Instead, States will be required to automatically designate all students who are unemployed and in the labor force as the cohort for which “entered employment” must be tracked. In addition, States will automatically designate all students who enter the program employed as the cohort for which “retained employment” must be tracked.

The revised text to the Implementation Guidelines with the proposed changes for the entered and retained employment measures is shown below in bold italics.

Proposed Changes to the Definition and Reporting of Entered Employment

Definition: Learner enters employment by the end of the first quarter after the program exit quarter. Employment is working in a paid, unsubsidized job or working 15 hours or more per week in an unpaid job on a farm or business operated by a family member or the learner. The exit quarter is the quarter when instruction ends, the learner terminates or has not received instruction 90 days, and is not scheduled to receive further instruction. A job attained while a learner is enrolled can be counted for entered employment and reported if the learner is still employed in the first quarter after exit from the program.

Applicable Population: Learners who are not employed at time of entry and in the labor force who exit during the program year.

Federal Reporting: States report the total number of unemployed learners in the workforce who enter employment and exit during the program year and the total number of learners in the workforce who are unemployed at entry who exit during the program year. Entered employment rate is computed by dividing these numbers.

Proposed Changes to the Definition and Reporting of Job Retention

Definition: Learner remains employed in the third quarter after program exit.

Applicable Population: Learners who, at time of entry, were not employed and in the workforce, who enter employment by the first quarter after exit quarter, and learners employed at entry.

Federal Reporting: States report the total number of learners in the applicable population who are employed in the third quarter after program exit and the total applicable population (i.e., the number of learners in the workforce who were unemployed at entry, and who entered employment; and learners who are employed at entry). The retained employment rate is computed by dividing these numbers.

States will be required to report data using the revised version of NRS Table 5 shown in the revised instrument for the program year beginning July 1, 2012.

2. Obtaining a Secondary Credential Measure

The NRS requires States to report attaining a secondary credential or passing the GED tests for students who set this outcome as a goal. States must track these students after they exit the adult education program to determine whether the students obtained the credential. OVAE will discontinue goal-setting in the collection of the other NRS follow-up measures (employment and postsecondary entry) because data indicate that local programs may not be effective in setting goals, thus resulting in measures of questionable utility.

Although goal-setting and reporting for the secondary credential measure appear to suffer less from inconsistent goal setting, to achieve consistency and simplify reporting, OVAE will eliminate goal-setting for the secondary credential attainment measure. States will instead be required to report the total number of enrolled students who take all the GED tests during the year and the number who pass them to calculate a pass rate. States with adult high schools will report the number of students in the NRS high Adult Secondary Education (ASE) level who obtain a high school diploma. States with an External Diploma Program (EDP) will report the number of students enrolled in the assessment phase who obtain a high school diploma.¹

The revised text to the Implementation Guidelines with the proposed changes for the measure of obtaining a secondary credential measure is shown below in bold italics.

Proposed Changes to the Definition and Reporting of Obtaining a Secondary Credential Measure

Definition: The learner obtains certification of attaining passing scores on GED tests, or the learner obtains a diploma or State-recognized equivalent, documenting satisfactory completion of secondary studies (high school or adult high school diploma).

Applicable Population: All enrolled learners who take all GED tests, or are enrolled in adult high school at the high ASE level, or are enrolled in the assessment phase of the EDP who exit during the program year.

Federal Reporting: States report the total number of learners in the applicable population (see above) and the number of learners who pass all GED tests or obtain secondary school diplomas. To compute a rate or percentage of attainment, the number of students receiving a secondary school diploma or GED is divided by the total number of learners in the applicable population who exit during the program year. Note that if a State has a policy officially recognizing attainment of a foreign language GED as receipt of a secondary school diploma or its recognized equivalent, the State may also report attainment of a foreign language GED in the NRS for adult literacy.

States will be required to report data using the revised version of NRS Table 5 shown in the revised instrument for the program year beginning July 1, 2012.

3. Entry into Postsecondary Education or Training Measure

The NRS requires States to report student entry into postsecondary education or training for the students who set this outcome as a goal. Local programs must follow up with these students after they exit the adult education program to determine whether they entered postsecondary education or training.

The data indicate that States may not be effective in setting the postsecondary entry goal. Over the last three program years, the national percentage of students with this goal averaged about 27 percent of students at the high adult secondary education (ASE) level and about 12 percent of all ASE students. Many States have reported less than 30 percent of students in ASE have postsecondary entry goals.

In addition, the number of students with postsecondary entry goals set and met over the last several years fluctuates greatly within individual States. This variation suggests instability in the data that is likely due to problematic goal-setting and data-collection practices rather than differences in the student population over the years. Consequently, it appears that the data on postsecondary entry are of limited use as a measure of accountability.

To correct this problem and thereby improve the quality of data, OVAE will discontinue the use of goal setting as a means for States to identify students to follow up to determine entry into postsecondary education or training. Instead, States will be required to designate all learners who have a GED, a secondary credential, or are enrolled in a class specifically designed for transitioning to postsecondary education (e.g., bridge program, college readiness), as the cohort for which “entry into postsecondary education” must be tracked. Students included in the cohort are those who passed the GED tests or earned a secondary credential while enrolled in adult education, enter with these credentials, or are enrolled in a class specifically designed for transitioning to postsecondary education.

In addition, OVAE proposes to extend the follow-up period for entry into postsecondary education to include the next program year (PY) after exit. For example, States can report entry into postsecondary education for a student who exits in PY 2012 any time from exit till the end of PY 2013. OVAE also clarifies that postsecondary education includes postsecondary occupational skill training and appropriate apprenticeship training, in addition to community colleges and four-year colleges.

The revised text to the Implementation Guidelines with the proposed changes for the entry into postsecondary education or training measure is shown below in bold italics.

Proposed Changes to the Definition and Reporting of Entry into Postsecondary Education or Training Measure

Definition: Learner enrolls in a postsecondary educational, occupational skills training program, or an apprenticeship training program that does not duplicate other services or training received, regardless of whether the prior services or training were completed.

Applicable Population: All learners who passed the GED tests or earned a secondary credential while enrolled in adult education, have a secondary credential at entry, or who are enrolled in a class specifically designed for transitioning to postsecondary education who exit during the program year.

Federal Reporting: The total number of learners who enter postsecondary education or a training program and the total number of students in the applicable population (see above) who exit during the program year are reported. Entry into postsecondary education or training can occur any time from the time of exit till the end of the following program year. To compute a rate of placement, the number of students enrolling in postsecondary education or training, through the end of the program year following the year of program exit, is divided by the total number of learners in the applicable population who exit during the program year.

States will be required to report data using the revised version of NRS Table 5 shown in the revised instrument for the program year beginning July 1, 2012.

2. Add new measures to report on (a) students’ highest educational level or degree attained and on (b) teacher experience and certification.

1. Students’ Highest Educational Level or Degree Attained

Adult education students have a wide variety of educational backgrounds. Many have been schooled exclusively in the United States, while a growing number of learners from foreign countries arrive with high school diplomas, college degrees, professional certifications, or no schooling at all. Data about the educational background of adult education students can be useful for local programs in curriculum development and student recruitment activities. States may also use this data for planning purposes or to inform broader policy discussions such as those that seek to address the career pathways of highly educated foreign professionals. At the Federal level these data can be useful in responding to national stakeholders and government officials who inquire about the educational attainment of the adult education students who enter our programs.

Beginning July 1, 2012, States will be required to report these data using a revised version of NRS Table 6, shown in the revised instrument.

B. Teacher Experience and Certification

The professional development and qualifications of teachers is of particular concern of OVAE and the States. Many States have implemented policies articulating the minimum requirements for adult education teachers and currently collect their own data on teacher educational attainment, certification, and experience for program planning and reporting purposes. These data often support State policy proposals regarding standards for professional credentials, inform the legislative process, and respond to public requests for information about teacher professionalism. States also use these data to plan appropriate professional development opportunities. Adult education researchers use the data to analyze a critical policy questions about teacher effectiveness and program performance. At the Federal level, these data can be useful in responding to national stakeholders and government officials who inquire about teacher standards and the credentials of adult education professionals.

Beginning July 1, 2012, States will be required to report these data using a revised version of NRS Table 7, shown in the revised instrument.

1 The “total students” included would be all students who take all five GED tests, or enroll in adult high school credit programs, or enroll in the EDP. The performance measure would be the number of students who passed all the GED tests, or received their high school diplomas, divided by this total.