Request For Clearance: National Science Foundation, Directorate of Education and Human Resources, Division of Graduate Education
Distance Monitoring Data Collection in Support of the Computer & Information Science & Engineering’s Pathways to a Revitalized Undergraduate Computing Education Program (CPATH)
This request for Office of Management and Budget (OMB) review asks for an initial clearance for a distance monitoring data collection in support of the Computer & Information Science & Engineering’s Pathways to a Revitalized Undergraduate Computing Education Program (CPATH), which is administered by the National Science Foundation (NSF)’s Directorate for Computer and Information Science and Engineering (CISE). This clearance is requested under the Directorate for Education and Human Resources (EHR) Generic Clearance, OMB 3145-0136, which expires on March 31, 2011. The EHR Generic Clearance includes collections of information about NSF’s education and training (E&T) activities.
As part of its mission CISE contributes to the education and training of the next generation of computer scientists and engineers. Through the CPATH program CISE is challenging its community partners – colleges, universities, and other stakeholders committed to advancing the field of computing and its impact – to transform undergraduate computing education on a national scale in order to meet the challenges and opportunities of a world where computing is essential to U.S. leadership and economic competitiveness across all sectors of society.
The use of computers has permeated and in many cases transformed almost all aspects of our everyday lives. As computer use becomes more important in all sectors of society, so does the need for preparation of a globally competitive U.S. workforce with knowledge and understanding of critical computing concepts, methodologies, and techniques. Thus, upgrading undergraduate computing education to keep abreast of the multitudes of rapid changes in computing is paramount for the U.S. economy and competitiveness.
The CPATH vision is of a U.S. workforce with the computing competencies and skills imperative to the Nation’s health, security, and prosperity in the 21st century. This workforce includes a cadre of computing professionals prepared to contribute to sustained U.S. leadership in computing in a wide range of application domains and career fields, and a broader professional workforce with knowledge and understanding of critical computing concepts, methodologies, and techniques.
To achieve this vision, CISE is calling for colleges and universities to work together and with other stakeholders (industry, professional societies, and other types of organizations) to formulate and implement plans to revitalize undergraduate computing education in the United States. The full engagement of faculty and other individuals in CISE disciplines will be critical to success. Common challenges are fluctuating enrollments in traditional computer science programs, changes and trends in workforce demographics, the imperative to integrate fast-paced computing innovations into the curriculum, and the need to integrate computing concepts and methodologies into the undergraduate curriculum at large. Goals and strategies must be developed to address these and other challenges. Successful CPATH projects will be systemic in nature, address a broad range of issues, and have significant potential to contribute to the transformation and revitalization of undergraduate computing education on a national scale.
NSF needs data about CPATH awards for project and program monitoring to fulfill policy and program reporting needs and to serve as preliminary work for future impact assessment and evaluation activities. The data collected as part of OMB 3145-0136 will allow NSF officials to document the overall program investment in individual alliances and make future funding and program policy decisions.
The purpose of the distance monitoring system is to collect baseline data for each existing CPATH project and to provide updated data in each subsequent year of the project. The CPATH PIs will be responsible for completing the data collection forms in the distance monitoring system.
The distance monitoring data collection form is included in appendix A. The screen shots for the online monitoring tool are provided in appendix B. The emails that will be sent to PIs are located in appendix C.
The information to be collected in this online system is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements; for measuring the implementation of this program, its projects, and strategic goals as laid out in NSF’s Strategic Plan; and for establishing a baseline for future program evaluations.
The primary purpose of this collection is program management, also known as program monitoring. This data collection activity is designed to track the design of the interventions, the strategies implemented, and the stakeholders engaged by CPATH awards to meet the objectives of the program. The information will be used by the CISE Directorate to administer and monitor the progress of the CPATH program. The findings will be used to recommend, among other things, administrative changes in program specifications, individual program focus and emphasis, and recruiting efforts.
The CPATH program will also use the data to fulfill reporting requirements. As part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected in the CPATH monitoring system may be used to report to these committees on program activities.
NSF is required to measure the attainment of its program, project, and strategic goals by the President’s Management agenda as represented by the Office of Management and Budget’s (OMB) Program Assessment Rating Tool (PART), by the Government Performance and Results Act (GPRA) of 1993, and by NSF’s Strategic Plan. Data collected in the CPATH monitoring system will help NSF management examine progress towards the Foundation’s goals and respond to these reporting requirements.
The data can also be used as a preliminary step to more in depth evaluation efforts, such as the sort of rigorous evaluations described in the May 2007 Report of the Academic Competitiveness Council, which was established by the Deficit Reduction Act of 2005 (P.L. 109-171) to serve as a multi-agency effort to identify federal STEM education programs and establish their effectiveness. The full ACC report can be accessed at http://www.ed.gov/about/inits/ed/competitiveness/acc-mathscience/index.html. The data will be used as a baseline for examining outcomes in a program evaluation that SRI International will also conduct.
The distance monitoring system will include one Web-based form that will cover all award types. This form will collect the following types of information:
Staff and Project Participant Characteristics: Names and job titles/academic positions of survey respondents (generally the project PIs) and staff; student ethnicity, race, gender, and disability status.
Project Implementation Characteristics: Financial support received from sources other than NSF; types of partners/collaborators; evaluation of products/practices.
Project Outputs: Counts of enrolled students (with demographic data); contributions to project goals/targets; outreach/dissemination activities; partner organizations/collaborative projects.
Changes may be made to collections in subsequent years based on input from NSF and lessons learned during the previous year’s collection.
EHR tends to favor Web-based systems because they can facilitate respondents’ data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough checking of all submitted data for completeness, validity, and consistency. Checks are performed as data are entered. Most invalid data cannot enter the system, and questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF. Web-based surveys employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus, and predefined charts and graphics. All of these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden.
The data for this monitoring effort will be collected by 508-compliant Web-based forms. The question format will be primarily quick-response checkboxes, with number and text boxes provided for the addition of specific data or outstanding examples. Respondents may save, leave and re-enter their data collection form as often as they desire and continue to change their responses until they submit their form. Additionally, since the collection is Web-based, minor changes in wording and displays can be easily made in response to user feedback.
This data collection will not duplicate other NSF efforts; comparable data are not collected on an annual basis for the CPATH program. The CPATH data collection will be coordinated with the NSF FastLane Project Reports system (OMB 3145-0058) to ensure that the two collections do not gather similar data. As much as possible, data from CPATH proposals and annual reports will be used to reduce the overall response burden. Additionally, aggregate data will be shared with NSF-funded researchers as appropriate, thereby minimizing the possibility that other researchers will duplicate these efforts in their own future collections.
No information is to be collected from small businesses.
First, without this information NSF will be unable to document the effectiveness or outcomes of the CPATH program. Second, without this information NSF will have no way of making systematic modifications to the CPATH program (e.g., adequacy of funding amount, duration of award, institutional supports needed). Third, without this information NSF will be unable to comply fully with the GPRA and PART reporting requirements or with the congressional mandate that the Foundation evaluate its science, technology, engineering, and mathematics (STEM) education programs. Fourth, without this information NSF will be unable to disseminate information to other projects and institutions about successful approaches to transform undergraduate computing education on a national scale with the goal of meeting the challenges and opportunities of a knowledge society where computing is essential to economic competitiveness and social welfare.
The data collection will comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register August 24, 2007, Volume 72, Number 164, page 48694. No comments were received.
During initial development of the data monitoring system, the PIs on CPATH awards will review the system, and their feedback on the data collection tool will be taken into account in further developing the system. Changes to the system after initial development will be informed by ongoing consultations with the respondents.
No payments or gifts will be provided to respondents.
The data collected will be available only to the respondents, NSF, and members of SRI International’s CPATH evaluation project team. Data will be processed according to Federal and State privacy statutes. To protect privacy, only composite data or graphical representations will be released to the public.
When respondents are presented with the first screen of the data collection form, they will be instructed as follows: “Information from this data collection system will be retained by the National Science Foundation, a federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). These are confidential files accessible only to appropriate National Science Foundation (NSF) officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested as “for general use,” will be made available to anyone outside of the National Science Foundation for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c.
The Web-based collection will be an annual activity of the CPATH program. Currently there are 60 CPATH awards and sites. The PIs on these awards will be required to submit data to the CPATH data collection system every year. Because new awards are likely to be added and some earlier awards will end, it is expected that the annual number of respondents will change somewhat over time.
The total number of annual person-hours for the first year of data collection is estimated at 180 hours. This burden is computed by multiplying the number of anticipated respondents (60) by the estimated response burden per person, based on pretesting of the data collection form by a sample of PIs.
Respondent Type |
Number of Respondents |
Burden Hours Per Respondent |
Annual Person-Hours |
PIs |
60 |
3 |
180 |
There is only one data collection form. The estimated annual burden is presented in section A.12.1 above.
The overall annualized cost to the respondents is estimated to be $6,926.40.
Respondent Type |
Number of Respondents |
Burden Hours Per Respondent |
Average Hourly Rate |
Total Annual Costs (rounded) |
PIs |
60 |
3 |
$38.48 |
$6,926.40 |
The estimated hourly rate for PIs is based on an annual survey conducted by the American Association of University Professors, data from which were published in The 2008-09 Chronicle Almanac, 55(1): 24, accessed at http://chronicle.com. According to results of this survey, the average salary of an associate professor across all types of doctorate-granting institutions (public, private, church-related) is $80,043 for the 2008–09 academic year. This calculates to an average hourly rate of $38.48 when divided by the 2,080 hours in a standard work year.
There will be no overall annual cost burden to respondents or record-keepers that results from the distance monitoring of the CPATH program other than the time spent responding to the data collection form.
It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. Furthermore, the majority of PIs are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, the PIs who will be the primary respondents for the CPATH data collection will make use of standard office equipment (e.g., computers) and Internet connectivity that are already required as startup and maintenance costs under the NSF GPG.
A.14. Estimates of Costs to the Federal Government
The table below shows the average annual cost to NSF for the CPATH distance data collection, by operational activity. The average annual cost of $194,000 is derived by summing the estimated budgets for the expected five-year term of the contract (base year + four option years) and dividing by five.
Operational Activity |
Average Annual Cost |
a. System development (includes initial development of the database and Web-based application and later changes requested by the program, e.g., increased reporting tools, additional validations) |
$93,120 |
b. System maintenance, updates, and technical support (updates required each year before opening the collection; maintenance required to keep the system current with technology, e.g., database servers, operating systems) |
$46,560 |
c. Data collection opening and support (e.g., online and telephone support to respondents, contacting respondents to encourage completion of the questions); reporting as defined by the Division of Graduate Education; and follow-up activities (e.g., providing data to other consultants) |
$54,320 |
Total Average Annual Cost to NSF |
$194,000 |
The total cost for the expected five-year term of the contract is $970,000.
As this application is for the first data collection, there are no changes in burden to report.
Data collection is scheduled to begin in February 2010. The PIs will have 45 days to enter the data for their award site. Extensions will be granted by NSF program officers as necessary. Once the data collection has been completed, agency staff can access the data through the online system as needed.
Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. SRI International, the contractor that manages the data collection Web site and database, is forbidden contractually from publishing results unless NSF has made a specific exception. In short, all products of the collections will be the property of NSF, and NSF will be the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information collected that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System (ODS) or simply a page on the NSF Web site) in which to publish.
The data from this collection will be used for internal review purposes, for monitoring the CPATH awards, and for reporting to Congress (e.g., the GPRA Annual Performance Plan). Reports to NSF management, PIs, and Congress dealing with characteristics and performance of the CPATH program will include statistical tables and charts generated from the database. As of this date, NSF has not set a timeline for publishing interim reports from this study.
Not applicable.
No exceptions apply.
| File Type | application/msword |
| File Title | Supporting Statement |
| Author | lthurgood |
| Last Modified By | nsfuser |
| File Modified | 2009-11-04 |
| File Created | 2009-11-04 |