Request For Clearance: National Science Foundation, Directorate of Education and Human Resources, Divisions of Research, Evaluation and Undergraduate Education
Program Monitoring System for the Robert Noyce Scholarship Program
Attachment G
This request for Office of Management and Budget (OMB) review asks for a renewal of clearance of the Program Monitoring System for the Robert Noyce Scholarship Program under the Directorate for Education and Human Resources (EHR) Generic Clearance, OMB 3145-0136, which will expire on January 31, 2008. The Noyce program is administered by the Division of Undergraduate Education (DUE) in EHR at the National Science Foundation (NSF).
The Robert Noyce Scholarship program, originally authorized under the National Science Foundation Authorization Act of 2002 (P.L. 107-368), responds to the critical need for K-12 teachers of science, technology, engineering, and mathematics (STEM) by encouraging talented STEM students and professionals to pursue teaching careers in elementary and secondary schools.
The original authorizing legislation stated that the "[NSF] Director shall carry out a program to award grants to institutions of higher education (or consortia of such institutions) to provide scholarships, stipends, and programming designed to recruit and train mathematics and science teachers. Such program shall be known as the 'Robert Noyce Scholarship Program'."(http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=107_cong_public_laws&docid=f:publ368.107). The program was reauthorized in 2007 under HR 2272, enacted as P.L. 110-69, the America COMPETES Act (http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=110_cong_public_laws&docid=f:publ069.110).
The program provides funding and programmatic support for STEM majors and STEM professionals to enter and complete teacher credentialing programs. Scholarship recipients are then required to complete two years of teaching in a high-need school district for each year of scholarship or stipend support. The program seeks to increase the number of K-12 teachers with strong STEM content knowledge.
The Noyce program is a key part of NSF’s efforts to achieve the strategic goal of Learning described in the FY 2006-2011 Strategic Plan (http://www.nsf.gov/pubs/2006/nsf0648/NSF-06-48.pdf). Primarily, the Noyce program addresses the strategic goals under NSF’s Learning mission. These include NSF’s goals of building strong foundations and fostering innovation to improve K-12 teaching learning, and evaluation in science and mathematics; and developing methods to effectively bridge critical junctures in STEM education pathways. Data collected from Noyce awards through the monitoring system are needed by NSF for project and program monitoring, to fulfill policy and program reporting needs, and to serve as preliminary work for future impact assessment and evaluation activities.
The Noyce Program Monitoring System will be the primary source of data on project activities. The monitoring system consists of a Web-based survey that will be completed by the principal investigator (PI) of each Noyce award (also called project or grantee).
Data collection instruments are included in appendix A.
The primary purpose of this collection is program management, also known as program monitoring. This data collection activity is designed to track the extent to which Noyce awards meet the objectives of the program. This information is used to administer and monitor the progress of the program in the various institutions. Those findings are used to recommend, among other things, administrative changes in program functions, level of fellowship support, individual program focus and emphasis, and recruiting efforts.
The Noyce program also uses the data to fulfill reporting requirements. NSF is required to measure the attainment of its program, project and strategic goals, as required by the President's Management agenda as represented by the Office of Management and Budget's (OMB) Program Assessment Rating Tool (PART); the Government Performance and Results Act (GPRA) of 1993; the Academic Competitiveness Council (ACC); and the NSF’s Strategic Plan. Data collected in the monitoring system help NSF management examine their progress towards these goals and respond to these reporting requirements. The data can also be used as a preliminary step in more detailed future evaluation efforts.
Under the Noyce monitoring system, each principal investigator (PI) of a Noyce award is required to provide annual data using the Web-based data collection system. The following is an overview of the types of information collected:
Data on scholarship or stipend recipients include:
Regarding other Noyce project participants the system will collect:
· Data on the discipline of each higher education faculty member involved in the project during the academic year
· Data on each of the post-secondary institutions involved in the project, including institution name and state
· Data on the programmatic activities conducted by the participating institutions, including mentoring, field experiences, and peer tutoring
· Aggregated data on applicants to individual projects, including number of scholarship/stipend applicants, number of recipients who received funding, number of recipients who graduated, and number of recipients who were certified
· Data on each school district involved in the project, including name, city and state, and number of schools involved in the project
· Project Baseline for each post-secondary institution, including number of STEM majors who graduated and became K-12 teachers one year prior to the award
As is required under OMB's Terms of Clearance for requests made under the EHR Generic Clearance, a summary crosswalk has been prepared to demonstrate how the requested information conforms to the scope of the EHR Generic. The Crosswalk is found in appendix B and provides the complete list of survey items or data elements. Appendix A provides screen shots of the web-based survey the respondents will use.
EHR tends to favor Web-based systems because they can facilitate respondents' data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough editing of all submitted data for completeness, validity and consistency. Editing is performed as data are entered. Most invalid data cannot enter the system, and questionable or incomplete entries are called to respondents' attention before they are submitted to NSF. Web based surveys employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus and predefined charts and graphics. All these features facilitate the reporting process, provide useful and rapid feedback to the data providers and reduce burden.
The data for this monitoring effort are collected by 508-compliant Web-based surveys. The survey structure allows respondents to move between a menu screen and a screen addressing individual topics. The question format is primarily quick-response checkboxes, with text boxes provided for the addition of specific, outstanding examples. Respondents may enter and leave their surveys as often as they desire and continue to change their responses until they submit their surveys. Respondents have access to an online glossary to assist them in understanding the specific meaning of terms in the context of these surveys. Additionally, since the collection is Web-based, minor changes in wording and displays can be easily made in response to user feedback.
This system does not duplicate other NSF efforts. Comparable data are not currently being collected on an annual basis for the Noyce program. In addition, the collection is coordinated with the NSF FastLane Project Reports system (OMB 3145-0058) to ensure that the two collections do not collect similar data. As much as possible, data from other NSF monitoring collections are used to pre-fill Noyce items, further minimizing overall response burden. Additionally, aggregate data are being shared with NSF-funded researchers as appropriate, thereby minimizing the possibility that other researchers will duplicate these efforts in their own future collections.
No information is to be collected from small businesses.
Without this information, NSF would be restricted in managing and reporting on the activities of awards in the Noyce program. Without this feedback, NSF would have no way of making systematic modifications to the program (e.g., adequacy of funding amount, duration of award, and institutional supports needed). These data will assist NSF in making informed decisions about future directions of the Noyce program. The information requested here is not available elsewhere.
Additionally, without this information NSF would find it difficult to meet GPRA and PART reporting requirements and would be unable to comply fully with congressional and presidential mandates that the Foundation asses its STEM education programs, and with the specific statutory requirements of the Noyce program.
The data collection will comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register August 24, 2007, Volume 72, Number 164, page 48692. No comments were received.
During the initial system development five PIs of Noyce awards reviewed the questions; their responses were taken into account in the development of the system. Additional changes in the system are informed by ongoing consultations with the respondents and Macro International Inc.(the contractor that designed the Web interface and database system) and user comments submitted during the collection period are also taken into consideration.No payments or gifts will be provided to respondents.
Data collected under this task are only available to the respondents, NSF, and the firms hired to manage the data and data collection software. Data are processed according to Federal and State privacy statutes. To protect privacy, only composite data or graphical representations will be released to the public.
For the collection covered by this clearance request, when respondents are presented with the first screen of the survey, they are additionally instructed as follows:
"Information from this data collection system will be retained by the National Science Foundation, a federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). These are confidential files accessible only to appropriate National Science Foundation (NSF) officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested as "for general use," will be made available to anyone outside of the National Science Foundation for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c."
The Noyce survey collects the name, telephone number, and e-mail address of each PI, and information on students including name and date of birth. These data are collected in order to monitor the award sites, to provide survey continuity, and to assess the success of the award programs. Respondents have the option of not providing information that they consider privileged, such as disability status, by marking the "not reported" option on the form. All information will be maintained in accordance with the requirements of the Privacy Act of 1974. Individualized data are provided only to Noyce program staff and to contractors authorized by NSF. Any public reporting of the data is in aggregate form.
The total average number of annual respondents is 75 and the average total annual person-hours is 1050. The Web-based collection is an annual activity of the Noyce program. There are currently 75 Noyce awards and data is collected from each award site; the respondent will be the PI of the award. We anticipate that new awards may be added, but at about the same rate that active awards expire; thus, on average, the number of respondents will remain constant over time.
The burden estimate is outlined below:
|
Type of |
Average Number of Respondents |
Annual Burden Hours Per Respondent |
Annual Person-Hours |
|
Project PIs |
75 |
14 hours |
1050 |
|
Total respondents |
75 |
Total estimated hours |
1050 |
The Noyce system consists of one form and, as mentioned above, respondents will be project PIs. The estimated total annual response burden is 1050 person-hours. The annual burden by form was calculated as follows:
|
Form Type |
Respondent Type |
Number of Respondents |
Burden Hours Per Respondent |
Total Person- Hours |
|
Principal Investigator survey |
Project PIs |
75 |
14 hours |
1050 |
|
Total |
|
75 |
|
1050 |
The overall annualized cost to the respondents is estimated to be $38,850. The following table shows the annualized estimates of costs to respondents. These estimated hourly rates are based on a report in the April 20, 2007, edition of The Chronicle of Higher Education (2007. “What Professors Earn.” The Chronicle of Higher Education, 53(33), Washington, D.C.: The Chronicle of Higher Education, Inc.). According to the report, the average salary of an associate professor across all types of doctoral-granting institutions (public, private, church-related) was $76,639. When divided by the number of standard annual work hours (2,080), this calculates to $37.00 per respondent hour.
|
Respondents |
Number of Respondents |
Hours per Respondent |
Average Hourly Rate |
Total Annual Costs |
|
Project PIs |
75 |
14 hours |
$37 |
$38,850 |
|
Total estimated costs |
|
|
|
$38,850 |
There is no overall annual cost burden to respondents or record-keepers that results from the distance monitoring of the Noyce program other than the time spent responding to the data collection instrument.
It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. Furthermore, the majority of PIs are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, PIs who are the respondents to the Noyce data collection make use of standard office equipment (e.g., computers) and Internet connectivity that are already required as a startup cost and maintenance cost under the NSF GPG. The information requested is typical of educational and research portfolios and would be maintained as part of normal practice.
Computing the annualized cost to NSF for the Noyce data collection was done by taking the budgets for 3 years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the Noyce data collection:
|
Operational Activities |
Cost Over 3 Years |
|
System development (includes initial development of the database and Web-based application and later changes requested by the program, e.g., increased reporting tools, additional validations) |
$310,250 |
|
System maintenance, updates, and technical support (system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$195,150 |
|
Data collection opening and support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of the questions), reporting (as defined by the Division of Undergraduate Education), and followup activities (e.g., providing data to other consultants) |
$200,200 |
|
3-Year Total for All Operational Activities |
$705,600 |
The annualized cost was computed as one-third of the total 3-year costs; thus, the annualized cost to NSF for the Noyce data collection is $235,200.
The previously reported total annual estimated response burden for this collection was 1,200 hours for 60 respondents; the current request for 1,050 hours for 75 respondents is decrease of 150 hours. This decrease is due to the fact that while the number of awards has increased slightly, most awards are now only entering one year of data; the first Noyce awards were made during FY 2002 and during the first year of data collection those awards that were active in previous years submitted data from more than one year. Now that the first year of data collection is complete, respondents will only enter one year of data at a time. This reduces the overall burden and simplifies data entry for respondents.
There have been no major changes in the instrument that would affect the burden. Minor changes in the data elements include small changes in question wording for clarification.
Data collection is scheduled to begin around November each year, and award sites will have approximately 90 days to enter data; extensions are granted by NSF program officers as necessary. Once the data collection has been completed, agency staff can access the data through the on-line system as needed.
Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. Macro International Inc., the contractor that manages the data collection Web site and database, is forbidden contractually from publishing results unless NSF has made a specific exception. In short, all products of the collections are the property of NSF and NSF is the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information collected that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System (ODS) or simply a page on the NSF Web site) in which to publish.
The data from this collection will be used for internal review purposes and to monitor the Noyce projects, as well as for reporting to Congress and for other mandated reporting requirements. Reports to NSF management, PIs, and Congress dealing with characteristics and performance of the Noyce program will include statistical tables and charts generated from the database. At this time, NSF has not set a timeline for publishing interim reports from this study.
Not Applicable
No exceptions apply.
The sample size is the entire universe of respondents. There are currently 60 PIs of Noyce awards and this number is expected to remain stable throughout the clearance period.
|
Population |
Estimated Universe Size |
Sample Size |
|
Noyce respondents |
75 |
75 |
This data collection uses a Web-based survey. Each Noyce award will be required to provide project information each year during the duration of their NSF funding. The data for post-support information, which relates to participants whose scholarship or stipend has ended but who continue to participate in the project, will be collected throughout the duration of the award and will continue to be requested following the award period. The solicitation states that awardees are responsible for tracking recipients to ensure that they complete their teaching requirement, so this data will be maintained by the respondents and is needed to determine whether the requirements of the program have been met. The latest program solicitation for the Noyce program can be found here: http://www.nsf.gov/pubs/2007/nsf07529/nsf07529.htm
NSF understands the limitations of this data collection, particularly in terms of using the data to determine program effectiveness. Data collected through the Noyce system are not used to determine the ultimate effectiveness of its STEM educational interventions, but are used in program planning and management, to report on agency activities and goals, and to lay the groundwork for future evaluations.
This data collection is a census, so no sampling is required.
Not Applicable
Not Applicable
Not Applicable
Not Applicable
The collection is part of reporting required of awardees. The first year of data collection had a 100 percent response rate and NSF anticipates that the rate for this collection will remain at this level. This is achieved by sending emails every three weeks to award sites that have not logged into the system, and by notifying all award sites still entering data when the system closing date is one week away. Approximately 60% of award sites receive at least one of these follow-up emails. Examples of emails that will be sent to the Noyce respondents are included in Appendix C; these are based on emails used in a similar monitoring system.
The questions in this collection were reviewed by Noyce PIs during the development of the system. In addition, many of the items and response categories follow formats that are already in use within other tasks, such as the IGERT Monitoring System, also cleared as part of the EHR Generic Clearance (OMB 3145-0136), including demographic questions, items on educational history, and questions about level of scholarship and stipend support.
Agency
Joan Prival, National Science Foundation, (703) 292-4635.
Deh-I Hsiung, National Science Foundation (703) 292-5153.
Bernice Anderson, National Science Foundation (703) 292-5151.
Contractors
Macro International Inc. of Bethesda, MD will be responsible for data collection and analysis under the direction of Lea Mesner, (301) 657-3077.| File Type | text/html |
| File Title | Supporting Statement |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |