REQUEST FOR RENEWAL OF EHR GENERIC CLEARANCE
OMB Approval number 3145-0136
Forms Clearance Package
Submitted by:
National Science Foundation
4201 Wilson Boulevard
Arlington, VA 22230
The National Science Foundation (NSF) funds research and education in science and engineering. NSF supplies grants, contracts, and cooperative agreements to more than 2,000 colleges, universities, and other eligible institutions, and provides graduate fellowships to individuals in all parts of the United States.
The Foundation accounts for about 20 percent of Federal support to academic institutions for basic research1. The Directorate for Education and Human Resources (EHR) is the unit within NSF that primarily is responsible for promoting rigor and vitality within the Nation's science, technology, engineering, and mathematics (STEM) education enterprise to further the development of a 21st Century STEM workforce. In addition, EHR provides support for research and implementation activities that may improve STEM learning and education at all levels, pre-school through postdoctoral, in traditional and non-traditional venues and among all citizens, residents and nationals.
This request for Office of Management and Budget (OMB) review asks for a regular or standard three-year renewal for the EHR Generic Clearance OMB 3145-0136 that expires on November 30, 2004.
The EHR Generic Clearance primarily is used for program and project planning, monitoring and management. Its component surveys or tasks typically collect data on inputs and outputs in order to generate outcome indicators that EHR needs to comply with the Government Performance and Results Act (GPRA). The data collected under this Clearance also serve as baseline data for program evaluations. EHR's Division of Research, Evaluation and Communication (REC) uses OMB 3145-0136 to coordinate the collection of information about NSF's Education and Training (E&T) portfolio of activities.
The EHR Generic Clearance relates to information collected under the NSF's Grant Proposal Guide OMB Control Number 3145-0058. Data gathered via OMB 3145-0058 is housed in NSF's main administrative database called the Proposal and Award System (PARS.) Most of the information in the EHR Generic Clearance, however, originates from specialized, custom collections. These individual collections (see attachments A-K) are designed to assist in management of specific programs, divisions, or multi-agency initiatives.
The scope of the EHR Generic Clearance covers information about activities that are funded or managed by EHR, and education and training programs and projects that are funded or managed by NSF's disciplinary directorates, offices, or programs. There are currently 14 previously approved collections under the existing clearance that will expire in November 2004. Three (3) of these collections will be ending upon the November 2004 expiration, making this request for extensions of 11 tasks.
The NSF Directorate of Education and Human Resources (EHR) is responsible for collecting, analyzing, evaluating, and communicating information on STEM education and human resource development activities, and for coordinating analytical and policy support for all NSF's Education and Training (E&T) portfolio.
In 1995 OMB requested that EHR establish a generic clearance in order to streamline its requests to OMB for information collections related to awards monitoring and OMB 3145-0136 was established. The generic clearance provides information for recurring studies for monitoring, managing, and communicating about NSF's investment in E&T programs, initiatives, and activities.
When the EHR Generic was cleared in 1998, the Terms of Clearance specified how individual packages would be handled. Those terms stated that "All . . . individual tasks associated with this generic . . . must be submitted to OMB for clearance prior to implementation. If approved those individual approvals will expire, at the latest, when this generic expires in 9/2001 . . . When NSF seeks to add additional tasks to 3145-0136 other than those previously mentioned, the additional request will be accompanied by an 83-C burden change sheet so that the appropriate burden total for the generic clearance can be changed accordingly. Further, each additional request shall contain a cover memo which describes why the specific task is appropriate to include in the generic. . . Consistent with past procedures under this generic clearance, submission of individual task are done informally (i.e., sent directly to the desk officer rather than to the docket library) and OMB will attempt to complete the review expeditiously."
The 2001 Terms of Clearance (TOC) further prescribed a "cross-walk that was provided by NSF on 11/6" and specifies that the cover memos submitted with new requests "should contain a similar crosswalk that details how the new questions fit into the 3 categories given." In addition, the 2001 TOC state that "NSF has agreed to consider this clearance to encompass only 'monitoring' surveys, and no program evaluations will be completed under this generic clearance. Evaluations will need to go through a full clearance review under the PRA. All monitoring studies must conform to the 3-category configuration explained in the memo of 10/24."
A renewal of the EHR Generic Clearance that allows continued collection of these data is requested. Many of the data collection instruments have similar structures, and while they seek information about different activities, they are often designed to collect information to allow for comparison and aggregation across activities. For example, EHR graduate training programs data elements fall into one of three major categories specified in the crosswalk in the 2001 Terms of Clearance:
Staff and participant characteristics
Project implementation characteristics
Project outputs
A crosswalk comparing the data collected across the task collections can be found in Appendix C.
The generic clearance also is designed for project self-evaluation that can yield performance data.
To provide effective analytical support, EHR needs current and reliable information about projects in NSF's E&T portfolio, the portfolio's impact, and its effectiveness. This information is particularly required to support studies and evaluations by EHR's Division of Research, Evaluation and Communication (REC), and studies by other NSF organizational units for project monitoring and effective administration. The information is retained in accordance with the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998) and collected by the EHR Generic Clearance (OMB 3145-0136). The Education and Training System of Records has several purposes, including:
Providing a source of information on demographic and educational characteristics and employment plans of participants in NSF-funded educational projects, in compliance with Foundation responsibilities to monitor scientific and technical resources.
Enabling NSF to monitor the effectiveness of NSF-sponsored projects and identify outcomes of projects funded under NSF awards for management evaluation, and for reporting to the Administration and Congress, especially under GPRA, 5 U.S.C. 306 and 39 U.S.C. 2801-2805.
Creating public use files (which contain no personally identifiable information) for research purposes
The EHR Generic Clearance and the Education and Training System of Records enable NSF staff members and third-party evaluators to collect and combine data from:
Surveys (paper, electronic (i.e., Web), and telephone)
Observations (i.e., site visits)
Face-to-face interviews
Focus groups
OMB 3145-0136 is focused on initiative-, division-, and program-specific quantitative and qualitative data collection activities. Data from these collections focus on activities and outcomes (i.e., the accomplishments of program grantees (projects) in terms of specific objectives). The data collections provide essential information for assessing progress toward NSF's major performance goals under GPRA (see Sources Cited). For example, several ongoing data collections seek to measure if programs have led to "improved mathematics, science, and technology skills for U.S. students at the K-12 level and for citizens of all ages" or have helped to promote "a diverse, internationally competitive and globally engaged workforce of scientists, engineers, and well-prepared citizens."
The EHR Generic Clearance data collections contribute to the general monitoring of project activities. Since EHR has limited staff members who must monitor hundreds of projects, a large-scale data collection is the only way in which these program officers can hope to track project activities. It is this unique opportunity to integrate pre-existing and newly generated data in a coherent and timely manner that makes the continuation of these data collections critically important.
The NSF uses for this information collection may slightly vary by the type of program or initiative subject (e.g., graduate education programs are significantly different than programs that serve undergraduate education purposes) to survey in an individual task. Yet, the overall or primary purpose of the collection is program and project management. NSF often terms individual tasks "program or project monitoring," because they collect standard information from each project funded by a particular NSF program. Other purposes of the information collection include, communication, compliance, program evaluation, audit, and research.
In part as a result of the information collected through OMB 3145-0136, there have been changes made in NSF's E&T programs portfolio. For example, NSF decided to sunset the Collaboratives for Excellence in Teacher Preparation (CETP) Program, i.e. no money was request by NSF to support new CETP projects. Information from the EHR Generic Collection regarding the CETP Program's impact had a significant influence on this decision. The CETP collection that continues (attachment C) is strictly for monitoring the awards remaining after the Program was sunset. While most of the uses are not as dramatic as eliminating a program, they are significant to the normal operation of the EHR Directorate and to the individual projects outside the Foundation.
Another important use of the data collection is for program accountability and communication, particularly in terms of GPRA. Information may be used for each EHR division's annual report. These annual reports, in turn, are used by NSF's leadership to write the GPRA Report to Congress or respond OMB's Program Assessment Rating Tool (PART). The findings are particularly important in GPRA because EHR has considerably more quantitative data than the rest of NSF. For example, the number of teachers receiving training from NSF is specifically required in the GPRA Plan and is collected in the Systemic Initiative (attachment A) program.
Monitoring can lead to changes in program guidelines, NSF funding levels to a particular project and may result in improved benefits to participants in NSF projects. Program evaluators usually review the monitoring data as part of designing an independent, third-party review or a simple descriptive analysis of an EHR program investment, such as the Louis Stokes Alliances for Minority Participation (LSAMP) program (Attachment J). More frequently, monitoring leads to corrections by the respondents to their project activities or by NSF regarding the funding levels provided to a particular project.
All of the existing collections included under this generic clearance request use advanced electronic information technology-that is, the Web and email-to minimize data duplication and respondent burden. Any new collections that will be submitted in the future are also expected to be either Web- or email-based. The collections included in this clearance package and their methods of data collection are shown in chart 1.
Attachment |
Collection title |
Method of data collection |
A |
Survey for Course, Curriculum, and Laboratory Improvement Program (CCLI), Division of Undergraduate Education |
Web |
B |
Collaboratives for Excellence in Teacher Preparation Distance Monitoring Data Collection (CETP) |
|
C |
Centers for Learning and Teaching Monitoring System (CLT) |
Web |
D |
Centers for Research Excellence in Science and Technology Monitoring System (CREST) |
Web |
E |
Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (CSEMS) |
Web |
F |
Division of Undergraduate Education Project Information Resource System (DUE-PIRS) |
Web |
G |
Systemic Initiatives Monitoring for Educational Systemic Reform (ESR) |
Web |
H |
Graduate Teaching Fellows in K-12 Education Distance Monitoring System (GK‑12) |
Web |
I |
Distance Monitoring System for the Division of Graduate Education Integrative Graduate Education and Research Training Program (IGERT) |
Web |
J |
Louis Stokes Alliances for Minority Participation (LSAMP) Distance Monitoring |
Web |
K |
Self-Evaluation Indicator System (SEIS) Historical Black Colleges and Universities Undergraduate Program (HBCU-UP) for Awardees |
EHR tends to favor Web-based systems because they can facilitate respondents' data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough editing of all submitted data for completeness, validity, and consistency. Editing is performed as data are entered. Most invalid data cannot be entered into the system, and questionable or incomplete entries are called to respondents' attention before they are submitted to NSF. In cases where data is collected from small populations via email, spreadsheet forms are sent via email to respondents who then enter data and return the completed form, also by email.
The EHR Generic Clearance Web-based surveys employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus, and predefined charts and graphics. All these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden.
All collections in the EHR Generic comply with Section 508, the 1998 amendment to the Federal Rehabilitation Act, which mandates that the electronic and information technology used by Federal agencies be made accessible to all people with disabilities.
The EHR Generic Clearance does not duplicate efforts undertaken by the Foundation, other federal agencies or other data collection agents.
For example, NSF grants require the submission of Annual and Final Project Reports in accordance with OMB 3145-0058. Recipients of NSF grants, such as Principal Investigators (PI), must create and submit annual and final project reports using NSF's nationally recognized FastLane web template. (For more information on FastLane, see http://www.fldemo.nsf.gov.) To minimize overall response burden, OMB 3145-0136 items are designed, so that they can be shared with or use the FastLane Projects Reports System Surveys. Specifically, all financial data on program funding are drawn from OMB 3145-0058, which covers applications submitted through the NSF FastLane system and the upcoming grants.gov.
Only two tasks, the Survey for Course, Curriculum, and Laboratory Improvement Program (attachment A) and the Division of Undergraduate Education Project Information Resource System (attachment F), collects data from small businesses, which usually are partnered with an academic or education institution. EHR anticipates that an average of only 36 small businesses will be affected annually, and that only a small amount of data will be collected from any small business organizations, with the total small business response burden being less than 1 percent of the total EHR Generic Clearance response burden. Full details can be found in the subtasks' individual clearance requests.
Data collected for the EHR Generic Clearance are used to manage programs, monitor projects, coordinate with Federal and non-Federal education partners, provide Congress with information about government-supported activities, and implement GPRA. In many cases, the data need to be collected annually to inform the NSF management and evaluation processes.
If the information is not collected, NSF will be unable to document the effectiveness and outcomes of its programs. It will not be able to meet its accountability requirements or assess the degree to which projects are meeting their goals. Moreover, NSF will unable to comply fully with the congressional mandate that the Foundation evaluate its STEM education programs.
All data collections will comply with 5 CFR 1320.6, with the exception of the Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (Attachment E), which asks respondents to submit data quarterly. See attachment E for more information on the frequency of this collection.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register June 7, 2004, Volume 69, Number 109, page 31846. No comments were received. A copy of the notice can be found at the end of this document.
EHR routinely consults with research and evaluation experts, PIs, and educators affected by EHR investments when developing collection instruments. The purpose of these consultations is to assess the relevance, availability, and clarity of items. As suggested by OMB guidelines, these consultations also enable EHR staff to obtain a reliable estimate of the respondent burden generated by new instruments. When a new task is added to the collection or when an existing task is modified to add new instruments each instrument is pretested with fewer than nine individuals and revised following debriefings with participating respondents.
All outside consultations are described within the context of the specific data collection tasks. In tasks conducted earlier under the EHR Generic Clearance, consultations have included knowledgeable outsiders such as representatives of REC contractors responsible for technical and evaluation tasks and Fellows who work at the Foundation as guests under programs such as the Einstein Fellows Program or the American Association for the Advancement of Science (AAAS) Washington Fellows Program.
To date no payments or gifts have been provided to respondents. There are no plans to provide incentives to respondents, since the value of program and project monitoring surveys is of value to both the respondent and the NSF.
In the event that OMB decides to widen the scope of the EHR Generic Clearance to cover the pre-testing of instruments to be used for complex that employ multiple methods of data collection (e.g., control groups of respondents who are not beneficiaries of NSF funding), incentives may be necessary.
Respondents are advised that any information on specific individuals is maintained in accordance with the Privacy Act of 1974. Every data collection instrument displays both OMB and Privacy Act notices.
Respondents are told that data collected for the EHR Generic Clearance are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to Federal and State privacy statutes. Detailed procedures followed by EHR for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users. Data submitted are used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c.
The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court, or party in court, or Federal administrative proceeding, if the government is a party.
In some cases, collections in the EHR Generic Clearance request information from respondents including name, address, Social Security number (SSN), date of birth, and grade point average. These data are collected in order to monitor the award sites and evaluate the success of the award programs. Information of this nature is also used to track recipients of funding and training. For example, in the IGERT survey (attachment I), trainees' SSNs are used as a tracking mechanism to permit followup studies that examine the long-term effect of the IGERT Program on individuals' success. However, in the IGERT collection and in all tasks that request SSN, SSN is a voluntary field. Indeed all items of a sensitive nature are voluntary. Respondents may choose not to provide information that they feel is privileged, such as SSN, address, or date of birth. Any individualized data that are collected are provided only to program staff and consultants conducting studies using the data as authorized by NSF. Any public reporting of data is in aggregate form.
The chart below shows which individual tasks include questions of a sensitive nature.
Attachment |
Collection title |
Contains Questions of Sensitive Nature |
A |
Survey for Course, Curriculum, and Laboratory Improvement Program (CCLI), Division of Undergraduate Education |
No |
B |
Collaboratives for Excellence in Teacher Preparation Distance Monitoring Data Collection (CETP) |
Yes |
C |
Centers for Learning and Teaching Monitoring System (CLT) |
No |
D |
Centers for Research Excellence in Science and Technology Monitoring System (CREST) |
Yes |
E |
Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (CSEMS) |
Yes |
F |
Division of Undergraduate Education Project Information Resource System (DUE-PIRS) |
No |
G |
Systemic Initiatives Monitoring for Educational Systemic Reform (ESR) |
No |
H |
Graduate Teaching Fellows in K-12 Education Distance Monitoring System (GK‑12) |
Yes |
I |
Distance Monitoring System for the Division of Graduate Education Integrative Graduate Education and Research Training Program (IGERT) |
Yes |
J |
Louis Stokes Alliances for Minority Participation (LSAMP) Distance Monitoring |
Yes |
K |
Self-Evaluation Indicator System (SEIS) Historical Black Colleges and Universities Undergraduate Program (HBCU-UP) for Awardees |
No |
As shown in Appendix B, and in Chart 3 below, the annual response burden for the 11 tasks cleared under OMB 3145-0136 is 39,188 hours (for 19,522 responses). Given the diversity of respondent types, the methods used to arrive at individual task burden estimates and are described in detail in attachments A-K.
Attachment |
Collection title |
Number of respondents |
Number of responses |
Annual hour burden |
A |
Survey for Course, Curriculum, and Laboratory Improvement Program (CCLI), Division of Undergraduate Education |
779 |
779 |
349 |
B |
Collaboratives for Excellence in Teacher Preparation Distance Monitoring Data Collection (CETP) |
130 |
130 |
4680 |
C |
Centers for Learning and Teaching Monitoring System (CLT) |
933 |
933 |
1,219 |
D |
Centers for Research Excellence in Science and Technology Monitoring System (CREST) |
14 |
14 |
1,022 |
E |
Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (CSEMS) |
12,400 |
13,200 |
3,200 |
F |
Division of Undergraduate Education Project Information Resource System (DUE-PIRS) |
1,800 |
1,800 |
1,200 |
G |
Systemic Initiatives Monitoring for Educational Systemic Reform (ESR) |
41 |
41 |
2900 |
H |
Graduate Teaching Fellows in K-12 Education Distance Monitoring System (GK‑12) |
130 |
130 |
2,990 |
I |
Distance Monitoring System for the Division of Graduate Education Integrative Graduate Education and Research Training Program (IGERT) |
1,700 |
1,700 |
6600 |
J |
Louis Stokes Alliances for Minority Participation (LSAMP) Distance Monitoring |
701 |
701 |
13,336 |
K |
Self-Evaluation Indicator System (SEIS) Historical Black Colleges and Universities Undergraduate Program (HBCU-UP) for Awardees |
94 |
94 |
1,692 |
|
Total |
18,722 |
19,522 |
39,188 |
NSF estimates approximately 10 new tasks will need to be cleared under the EHR Generic Clearance during the next 3 years, dependent on budgetary limitation and Congressional mandates. The overall response burden in any year should not exceed 50,000 hours. The burden associated with each new task will be outlined in the individual requests that will be submitted to OMB with a burden change request form, OMB Form 83-C.
Below is an example that shows how the hour burden was estimated for the CLT Monitoring System, Attachment C:
The estimated annual response burden is 1,219 person-hours. The respondents are graduate students, post-doctoral fellows, CLT faculty, professional developers/other educators, and evaluators. Respondents provide answers to the instruments just once a year. The burden estimate is based upon the sum of the number of participants during the 2003-04 academic year, in addition to an estimated number of participants from three newly-funded Centers. Each Center will also have one evaluator complete the aggregate data. The calculations used to determine overall response burden are shown in Chart 4 below.
Chart 4: Calculations used to estimate overall response burden for the CLT online collection |
|||
Respondent Type |
Number of Respondents |
Hours Per Respondent Type |
Annual Person Hour Total |
Graduate students |
400 |
0.50 |
200 |
Post-doctorates |
45 |
0.50 |
23 |
Faculty members |
235 |
0.75 |
176 |
Professional Developers/Other Educators |
200 |
0.75 |
150 |
Evaluators |
13 |
50.00 |
650 |
Exiting Participants |
40 |
0.50 |
20 |
Total |
933 |
N/A |
1,219 |
A.12.2. Hour Burden Estimates by Each Form and Aggregate Hour Burdens
Details on the burdens of each form can be found in the task clearances. The chart below is an example of how this burden was estimated for the IGERT Monitoring System, Attachment I.
Chart 5: Calculations used to estimate burden by form for the IGERT Monitoring System |
||||
Form Type |
Respondent Type |
Number of Respondents |
Burden Hours Per Respondent |
Total Person- Hours |
Principal Investigator survey |
Project PIs |
100 |
34 hours |
3,400 |
Trainee Survey |
Trainees |
1,600 |
2 hours |
3,200 |
Total |
|
1,700 |
|
6,600 |
As shown in Appendix B, the total annual cost to respondents generated by the 11 ongoing data collections is currently estimated to be $856,633. Below is an example of the method used to calculate cost burden for the ESR Monitoring System, Attachment G.
Chart 6: Calculations used to estimate the cost to respondents for the ESR Monitoring System |
||||
Respondent |
Number of Respondents |
Hours per Respondent |
Average Hourly Rate |
Total |
SSI PIs |
2 |
46 |
$33 |
$3,036 |
USP PIs |
39 |
72 |
$33 |
$92,664 |
Total |
|
|
|
$95,700 |
The costs to respondents generated by additional data collections will be described in the individual task request for each data collection and reported on the OMB Form 83-C.
There is no overall annual cost burden to respondents or record-keepers that results from the EHR Generic Clearance other than the time spent responding to surveys that are described in specific detail under A.12 within the attached individual task justifications (attachments A-K).
It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. Furthermore, the majority of respondents are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system, or via the upcoming grants.gov. Thus, principal investigators, K-12 administrators, faculty members, and college students, who are the primary respondents to the individual data collections tasks within the EHR Generic Clearance, make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance cost under OMB 3145-0058, and free software (e.g., Netscape or Microsoft Explorer) to respond.
As shown in Appendix B, the total annual cost to the federal government of the 11 ongoing data collections is currently estimated to be $1,382,721. Details of the costs of each task can be found in Appendix B.
Below is an example of the costs to the federal government from the CREST data collection, Attachment D:
Computing the annualized cost to NSF for the CREST data collection was done by taking the budgets for 3 years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the CREST data collection:
Chart 7: Costs to the Federal Government for the CREST data collection
Operational Activities |
Cost Over 3 Years |
System Development (includes initial development of the database and Web-based application, and later changes requested by the program-e.g., increased reporting tools, additional validations) |
$198,500 |
System Maintenance, Updates, and Tech Support (system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$117,500 |
Data Collection Opening and Support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of the questions), Reporting (as defined by HRD), and Followup activities (e.g., providing data to other consultants) |
$80,000 |
3-Year Total for All Operational Activities |
$396,000 |
The annualized cost was computed as one-third of the total 3-year costs; thus, the annualized cost to NSF for the CREST data collection is $132,000.
More details on the costs of existing tasks can be found in the individual task clearances. The costs to the government generated by future data collections will be described in the clearance request for each data collection.
During the last 3 years, in accordance with OMB's 2001 Terms of Clearance, NSF has requested clearance for both:
New (also called additional) collections as they are formulated
"Other individual tasks associated with this generic" (old or previously cleared tasks)
The current inventory numbers at OMB for the EHR Generic package cover 14 individual collection tasks. The OMB inventory records show a total number of respondents of 16,517 and total hours of 49,733.
During the extensive document review to prepare this package NSF estimates that the 49,733 number is too high--off by something less than 5,700 hours. We believe this error primarily derives from an OMB Notice of Action for approved burden change in March 2003. IGERT monitoring surveys requested a change in burden. The burden change requested a difference of 690 hours (over the existing inventory) that brought that task to 6,600 hours total. OMB, however according to the notice of action recorded the requested difference as 6, 600 hours.
For this renewal, three of the previous tasks are sunsetting, so we request that OMB approve the 11 individual tasks as requested and set their expiration to coincide with the EHR Generic Clearance's expiration in 2007. This renewal requests 18,772 total respondents and 39,188 total hours; details can be found in Appendix B. The change in burden is due to shifts in the number of respondents and small adjustments in the data requested. The chart below shows the changes in burden in the individual tasks:
Attachment |
Collection title |
Previously Cleared Burden |
Currently Requested Burden |
Change in Burden |
A |
Survey for Course, Curriculum, and Laboratory Improvement Program (CCLI), Division of Undergraduate Education |
362 |
349 |
-13 |
B |
Collaboratives for Excellence in Teacher Preparation Distance Monitoring Data Collection (CETP) |
5370 |
4680 |
-690 |
C |
Centers for Learning and Teaching Monitoring System (CLT) |
1116 |
1219 |
103 |
D |
Centers for Research Excellence in Science and Technology Monitoring System (CREST) |
255 |
1022 |
767 |
E |
Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (CSEMS) |
2134 |
3200 |
1066 |
F |
Division of Undergraduate Education Project Information Resource System (DUE-PIRS) |
1200 |
1200 |
0 |
G |
Systemic Initiatives Monitoring for Educational Systemic Reform (ESR) |
6352 |
2900 |
-3452 |
H |
Graduate Teaching Fellows in K-12 Education Distance Monitoring System (GK‑12) |
3956 |
2990 |
-966 |
I |
Distance Monitoring System for the Division of Graduate Education Integrative Graduate Education and Research Training Program (IGERT) |
6600 |
6600 |
0 |
J |
Louis Stokes Alliances for Minority Participation (LSAMP) Distance Monitoring |
12788 |
13336 |
548 |
K |
Self-Evaluation Indicator System (SEIS) Historical Black Colleges and Universities Undergraduate Program (HBCU-UP) for Awardees |
1692 |
1692 |
0 |
Not being renewed |
Division Of Human Resources Development Project Information Resource System (HRD-PIRS) |
210 |
0 |
-210 |
Not being renewed |
Division of Research, Evaluation and Communication (REC) Research Data Collection |
234 |
0 |
-234 |
Not being renewed |
Graduate Research Traineeships Program Distance Monitoring Survey System Data Collection (GRT) |
1782 |
0 |
-1782 |
|
NSF Burden Estimates Totals/OMB Estimated Totals
|
44,051/49,733 |
39,188 |
-4863/ -10,454 |
According to the OMB inventory records, the total change of burden is a decrease of 10, 454 hours. As the table above shows NSF can account for 4,863 of those hours. The remaining 5,682 hours date primarily from the 2003 approval of IGERT.
Changes in the hour burden are matched by changes in the number of respondents. The chart below shows the changes in total number of respondents.
Attachment |
Collection title |
Previously Cleared Number of Respondents |
Currently Requested Number of Respondents |
Change in Number of Respondents |
A |
Survey for Course, Curriculum, and Laboratory Improvement Program (CCLI), Division of Undergraduate Education |
889 |
779 |
-110 |
B |
Collaboratives for Excellence in Teacher Preparation Distance Monitoring Data Collection (CETP) |
150 |
130 |
-20 |
C |
Centers for Learning and Teaching Monitoring System (CLT) |
1078 |
933 |
-145 |
D |
Centers for Research Excellence in Science and Technology Monitoring System (CREST) |
390 |
14 |
-376 |
E |
Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (CSEMS) |
8200 |
12400 |
4200 |
F |
Division of Undergraduate Education Project Information Resource System (DUE-PIRS) |
1800 |
1800 |
0 |
G |
Systemic Initiatives Monitoring for Educational Systemic Reform (ESR) |
57 |
41 |
-16 |
H |
Graduate Teaching Fellows in K-12 Education Distance Monitoring System (GK‑12) |
1304 |
130 |
-1174 |
I |
Distance Monitoring System for the Division of Graduate Education Integrative Graduate Education and Research Training Program (IGERT) |
1700 |
1700 |
0 |
J |
Louis Stokes Alliances for Minority Participation (LSAMP) Distance Monitoring |
278 |
701 |
423 |
K |
Self-Evaluation Indicator System (SEIS) Historical Black Colleges and Universities Undergraduate Program (HBCU-UP) for Awardees |
94 |
94 |
0 |
Not being renewed |
Division Of Human Resources Development Project Information Resource System (HRD-PIRS) |
300 |
0 |
-300 |
Not being renewed |
Division of Research, Evaluation and Communication (REC) Research Data Collection |
225 |
0 |
-225 |
Not being renewed |
Graduate Research Traineeships Program Distance Monitoring Survey System Data Collection (GRT) |
924 |
0 |
-924 |
|
Total Number of Respondents & Total Change
|
17,389 |
18,722 |
1333 |
In future years, the burden will be affected by the deletion and addition of some subtasks and respondents. NSF will notify OMB whenever there are significant changes to the burden.
Often changes in burden are due to the additional of new items to previously cleared surveys. The chart below indicates which tasks in this clearance have had major items added since their last OMB clearance. More details can be found in individual clearances.
Attachment |
Collection title |
New Items/Modules Added |
A |
Survey for Course, Curriculum, and Laboratory Improvement Program (CCLI), Division of Undergraduate Education |
Yes (National Dissemination survey) |
B |
Collaboratives for Excellence in Teacher Preparation Distance Monitoring Data Collection (CETP) |
Yes (Questions 2 and 3) |
C |
Centers for Learning and Teaching Monitoring System (CLT) |
Yes (Module 4) |
D |
Centers for Research Excellence in Science and Technology Monitoring System (CREST) |
Yes (Items on Presentations and Contributors) |
E |
Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (CSEMS) |
No |
F |
Division of Undergraduate Education Project Information Resource System (DUE-PIRS) |
No |
G |
Systemic Initiatives Monitoring for Educational Systemic Reform (ESR) |
No |
H |
Graduate Teaching Fellows in K-12 Education Distance Monitoring System (GK‑12) |
Yes (Items on Additional Funding Sources and Fellow Contact Information) |
I |
Distance Monitoring System for the Division of Graduate Education Integrative Graduate Education and Research Training Program (IGERT) |
Yes (Item on postdoctoral and undergraduate training) |
J |
Louis Stokes Alliances for Minority Participation (LSAMP) Distance Monitoring |
Yes (new module) |
K |
Self-Evaluation Indicator System (SEIS) Historical Black Colleges and Universities Undergraduate Program (HBCU-UP) for Awardees |
No |
Like many agencies, NSF no longer relies on formal (i.e., traditional) publication methods and publication formats. News media advisories, notices of funding opportunities for colleges and universities, and results from survey collections are all examples of the types of publications that NSF regularly publishes without putting ink to paper.
For content considered to be authored by or at NSF's request by a third party, the agency rarely uses paper to publish the information. NSF publishes most documents electronically only using the agency's Web site, from requests for proposals to evaluation or statistical reports using an archive called an On-Line Document System (ODS).
In addition NSF runs a Custom News Service, an email and Web-based alert service, that sends documents newly published in the ODS (e.g., vacancy announcements, calls for proposals, statistical reports) to subscribers. Subscribers receive electronically those NSF documents of interest and not the agency's entire publications line.
The other major venue for NSF publications is FastLane. The NSF FastLane system collects and publishes information from NSF's clients (i.e., applicants for funding to NSF) using the Web. When an applicant's proposal has been funded, that applicant's name and other key data are published on NSF's Web site. Each week the FastLane Web site publishes a list of new awards using data gathered from the application process.
Like NSF itself, the scope of publication plans and practices by the OMB 3145-0136 EHR Generic Clearance has a dual nature. Some individual collections produce formal products (e.g., analytical reports) that can be published by NSF's ODS. Some collections produce only the respondents' replies that are posted verbatim on the EHR share of the NSF Web site for anyone to download.
Most of what the EHR Generic Clearance OMB 3145-0136 collects, however, is not published as a stand-alone product, because the data are an input to how NSF manages and measures its performance as an agency. NSF's GPRA Performance Report or an individual division's annual report to the NSF Director uses information from OMB 3145-0136 to report to Congress. This is an annual cycle.
Most of these tasks are the work of third-party contractors that deliver 1) analytical reports, 2) the raw data from the collections, or both. Third parties are contractually forbidden from publishing results unless NSF has made a specific exception. In short, all products of the collections are the property of NSF. After the products are delivered, NSF determines whether the quality of the products deserves publication verbatim by NSF; i.e., NSF typically is the exclusive publisher of the information collected by OMB 3145-0136. Often it is only after seeing the quality of the information the collection delivers that NSF decides the format (raw or analytical) and manner (in the ODS or simply a page on the NSF Web site) in which to publish.
EHR recurring studies are done to monitor, manage, and communicate with and about the clients funded by NSF's investment in education and training. In most cases the primary purpose for each recurring study is program management. These studies generate data that enable both NSF and the funded education and training projects to improve management and performance. Typically, recurring studies generate information that NSF uses as inputs to other reports (e.g., the GPRA Performance Plan), and therefore EHR cites no specific publication plans other than internal or general use to meet congressional reporting requirements.
There are, however, several collections within the EHR Generic Clearance that do, as previously approved, directly publish raw data from an individual collection using the NSF Web site. The model being employed is DUE's Project Information Resource System (PIRS) (attachment F). DUE's PIRS collects information from grantees of the Division and instantaneously publishes their verbatim response to the EHR Web site. There is an on-line system (at https://www.ehr.nsf.gov/PIRS_PRS_Web/Search/default.asp) that allows anyone to generate customized reports using data collected by the PIRS system.
REC uses data from recurring studies to provide information that can be mined for program evaluation purposes, such as identifying best practices in the education of graduate and undergraduate students or as a baseline for summative evaluation reports. In the past, using data in part, but not exclusively, from OMB 3145-0136, the following evaluative or descriptive analysis research reports have been produced:
A Description and Analysis of Best Practice Finding of Programs promoting participation of underrepresented undergraduate student in Science, Mathematics, Engineering and Technology (Westat) (NSF 01-31) (http://www.nsf.gov/pubsys/ods/getpub.cfm?nsf0131).
Summary Report on the Impact Study of the National Science Foundation's Program for Women and Girls, December 2000, (The Urban Institute) (NSF 01-27) (http://www.nsf.gov/pubsys/ods/getpub.cfm?nsf0127)
At this time, NSF has no set timeline for publishing monitoring or evaluation reports from these recurring studies, but plans that a summary or descriptive report be produced within 2 years of completion of the data collections for each recurring study.
Not applicable.
No exceptions apply.
In keeping with the original 1995 request and subsequent 1998 and 2001 OMB renewed approvals, the EHR Generic Clearance's (OMB 3145-0136) goal is a portfolio of individual collections used to count and describe the universe of NSF-funded or -partnered education and training projects. The statistical method employed in all eleven task collections is that of a census of NSF funded projects. Some projects have only one respondent type, typically a Principal Investigator (PI), others have several types of respondents
Data collection for the tasks involve all awardees in the programs involved. The chart below shows the total universe and sample size for each of the tasks.
Attachment |
Collection title |
Universe of Respondents |
Sample Size |
A |
Survey for Course, Curriculum, and Laboratory Improvement Program (CCLI), Division of Undergraduate Education |
779 |
779 |
B |
Collaboratives for Excellence in Teacher Preparation Distance Monitoring Data Collection (CETP) |
130 |
130 |
C |
Centers for Learning and Teaching Monitoring System (CLT) |
933 |
933 |
D |
Centers for Research Excellence in Science and Technology Monitoring System (CREST) |
14 |
14 |
E |
Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (CSEMS) |
12,400 |
12,400 |
F |
Division of Undergraduate Education Project Information Resource System (DUE-PIRS) |
1800 |
1800 |
G |
Systemic Initiatives Monitoring for Educational Systemic Reform (ESR) |
41 |
41 |
H |
Graduate Teaching Fellows in K-12 Education Distance Monitoring System (GK‑12) |
130 |
130 |
I |
Distance Monitoring System for the Division of Graduate Education Integrative Graduate Education and Research Training Program (IGERT) |
1,700 |
1,700 |
J |
Louis Stokes Alliances for Minority Participation (LSAMP) Distance Monitoring |
701 |
701 |
K |
Self-Evaluation Indicator System (SEIS) Historical Black Colleges and Universities Undergraduate Program (HBCU-UP) for Awardees |
94 |
94 |
|
Total
|
18,772 |
18,772 |
The data collections in this generic clearance use either Web- or email-based survey. Each respondent will provide answers once a year, with the exception of respondents to the CSEMS survey (attachment E), who enter data each semester/quarter, for an average of three times a year.
Each of the eleven task collections is a census, in which the sample size is the universe. Details on the size of the universe in each collection are included in individual clearances.
Not applicable.
Not applicable.
Not applicable.
Not applicable.
All of the task collections in this generic clearance are a part of the reporting required of awardees, so a high response rate is expected. The table below shows the expected response rates for each of the individual tasks.
Attachment |
Collection title |
Response Rate |
A |
Survey for Course, Curriculum, and Laboratory Improvement Program (CCLI), Division of Undergraduate Education |
80 percent |
B |
Collaboratives for Excellence in Teacher Preparation Distance Monitoring Data Collection (CETP) |
100 percent |
C |
Centers for Learning and Teaching Monitoring System (CLT) |
100 percent |
D |
Centers for Research Excellence in Science and Technology Monitoring System (CREST) |
100 percent |
E |
Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (CSEMS) |
90 percent |
F |
Division of Undergraduate Education Project Information Resource System (DUE-PIRS) |
50 percent |
G |
Systemic Initiatives Monitoring for Educational Systemic Reform (ESR) |
100 percent |
H |
Graduate Teaching Fellows in K-12 Education Distance Monitoring System (GK‑12) |
100 percent |
I |
Distance Monitoring System for the Division of Graduate Education Integrative Graduate Education and Research Training Program (IGERT) |
100 percent |
J |
Louis Stokes Alliances for Minority Participation (LSAMP) Distance Monitoring |
100 percent |
K |
Self-Evaluation Indicator System (SEIS) Historical Black Colleges and Universities Undergraduate Program (HBCU-UP) for Awardees |
100 percent |
Principal investigators are responsible for ensuring that other individual involved in the project submit all necessary data, and in many cases have access to status information on the Web-based systems indicating whether or not individual respondents in their projects have completed their data entry. In addition, EHR staff also have access to on-line monitoring sections of many of the Web-based systems and can check the status of reporting. A series of emails and phone calls are also used to follow-up with respondents and ensure that all necessary data is collected. See individual task collections for examples of the follow-up emails that are sent and more specific information on how response rates are supported.
All of the collections for which clearance is being requested are currently in operation and have been tested both before initial implementation and throughout the data collection. The LSAMP Monitoring system, for example, has been operational since 1995. Input on this system is continually received from users and their suggestions are implemented as the system is upgraded. Other test methods used by the various collections in the EHR Generic include feedback from PIs both as data is collected and during meetings and conferences, review by NSF staff, and testing performed by the system developer. Many systems are based on data collection methods currently used by other NSF groups, and many of the items and response categories follow formats that are already in place.
The following individuals were consulted on the EHR Generic Clearance:
Agency Unit
Mary Sladek (703-292-5152) and William Neufeld (703-292-5148), Division of Research, Evaluation and Communication, National Science Foundation, 703-292-5150
Mark Klawiter (703-292-5155), Einstein Fellow
Contractor or Grantee
Lea Mesner, ORC Macro, (301) 657-3077.
The following table shows the individuals involved in each task:
Attachment |
Collection title |
NSF Agency Unit |
Contractor or Grantee |
A |
Survey for Course, Curriculum, and Laboratory Improvement Program (CCLI), Division of Undergraduate Education |
Herb Levitan, (703) 292-4627 |
--- |
B |
Collaboratives for Excellence in Teacher Preparation Distance Monitoring Data Collection (CETP) |
Joan Prival, 703) 292-4635 |
Lea Mesner, ORC Macro, (301) 657-3077. |
C |
Centers for Learning and Teaching Monitoring System (CLT) |
Conrad Katzenmeyer, (703) 292-5150 |
Gary Silverstein, Westat, (301) 251-2244. |
D |
Centers for Research Excellence in Science and Technology Monitoring System (CREST) |
Victor Santiago, (703) 292-4673; William Neufeld, (703) 292-5148 |
Lea Mesner, ORC Macro, (301) 657-3077. |
E |
Survey Form for the Division of Undergraduate Education Computer Science, Engineering, and Mathematics Scholarships Program (CSEMS) |
Duncan McBride, 703-292-4630 |
--- |
F |
Division of Undergraduate Education Project Information Resource System (DUE-PIRS) |
Herb Levitan, (703) 292-4627 |
--- |
G |
Systemic Initiatives Monitoring for Educational Systemic Reform (ESR) |
Celestine Pea, (703) 292-8690 |
Lea Mesner, ORC Macro, (301) 657-3077. |
H |
Graduate Teaching Fellows in K-12 Education Distance Monitoring System (GK‑12) |
Terry Woodin, (703) 292-8697 William Neufeld, (703) 292-5148 |
Lea Mesner, ORC Macro, (301) 657-3077. |
I |
Distance Monitoring System for the Division of Graduate Education Integrative Graduate Education and Research Training Program (IGERT) |
Debasish Dutta, (703) 292-5304 |
Lea Mesner, ORC Macro, (301) 657-3077. |
J |
Louis Stokes Alliances for Minority Participation (LSAMP) Distance Monitoring |
A. James Hicks, (703) 292-4668 |
Lea Mesner, ORC Macro, (301) 657-3077. |
K |
Self-Evaluation Indicator System (SEIS) Historical Black Colleges and Universities Undergraduate Program (HBCU-UP) for Awardees |
Victor Santiago (703) 292-4673 and Jessie DeAro (703) 292-5350 |
Jason J. Kim and Linda M. Crasco, Systemic Research, Inc. (781) 278-0300 |
1 National Science Foundation, Division of Science Resources Statistics, Federal Obligations for Researchto Universities and College by Agency and Detailed Field of Science and Engineering: Fiscal Years 1973-2002.
| File Type | application/msword |
| File Title | REQUEST FOR RENEWAL OF EHR GENERIC CLEARANCE |
| Author | SPLIMPTO |
| Last Modified By | SPLIMPTO |
| File Modified | 2004-10-28 |
| File Created | 2004-10-28 |