Supporting Statement (3145-0136)

 

 

REQUEST FOR CLEARANCE

NATIONAL SCIENCE FOUNDATION

DIRECTORATE FOR EDUCATION AND HUMAN RESOURCES

DIVISION OF HUMAN RESOURCE DEVELOPMENT

CENTERS OF RESEARCH EXCELLENCE IN SCIENCE AND TECHNOLOGY DISTANCE MONITORING SYSTEM

Attachment A

Section A

Introduction

 

This request for Office of Management and Budget (OMB) review asks for a renewal of clearance of the distance monitoring data collection for the National Science Foundation (NSF) Division of Human Resource Development (HRD) Centers of Research Excellence in Science and Technology (CREST) program under the Directorate for Education and Human Resources (EHR) Generic Clearance OMB 3145-0136 that expires on January 31, 2008. The EHR Generic Clearance is for the collection of monitoring data on education and training activities supported by the NSF.

A.1. Circumstances Requiring the Collection of Data

The CREST program (http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=6668&org=HRD&sel_org=HRD&from=fund ) was initiated in 1987 to enhance the research capabilities of minority-serving institutions through the establishment of centers that effectively integrate education and research. CREST promotes the development of new knowledge, enhancements of the research productivity of individual faculty, and an expanded presence of students historically underrepresented in science, technology, engineering, and mathematics (STEM) disciplines. The program does this by

 

·        Developing outstanding research centers that integrate education with research to produce new knowledge and increase the number of underrepresented minorities with Ph.D.s in STEM fields

·        Making substantial resources available to upgrade the research and education capabilities of the most productive minority institutions

·        Enabling faculty at these institutions to cooperate with other science and engineering activities

·        Building bridges for minority student career development through alliances with business, government laboratories, and other universities

 

The CREST program is a key part of NSF’s efforts to achieve the strategic goals of Discovery and Learning described in the FY 2006-2011 Strategic Plan (http://www.nsf.gov/pubs/2006/nsf0648/NSF-06-48.pdf). Primarily, the CREST program addresses the strategic goals under NSF’s Learning mission. These include NSF’s goals of developing methods to effectively bridge critical junctures in STEM education pathways; preparing a diverse, globally engaged STEM workforce; and integrating research with education and building capacity.

Data collected from CREST centers through the monitoring system are needed by NSF for project and program monitoring, to fulfill policy and program reporting needs, and to serve as preliminary work for future impact assessment and evaluation activities.

A.2. Purposes and Uses of the Data

The information collected in this task is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements; for measuring attainment of NSF's program, project and strategic goals as laid out in NSF’s Strategic Plan; and as a
baseline for future program evaluations.

The primary purpose of this collection is program management, also known as program monitoring. This data collection activity is designed to track the extent to which CREST projects meet the objectives of the program. Within the HRD division, this information is used to administer and monitor the progress of the program. The findings are used to recommend, among other things, administrative changes in program functions, level of award support, individual program focus and emphasis, and recruiting efforts. 

 

The CREST program also uses the data to fulfill reporting requirements. As a part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected in the CREST monitoring system may be used to report to these committees on program activities. The materials developed for the September 2007 CREST COV included multiple years of summary data about the program that had been collected in the CREST monitoring system In addition, NSF is required to measure the attainment of its program, project and strategic goals by the President's Management agenda as represented by the Office of Management and Budget's (OMB) Program Assessment Rating Tool (PART), by the Government Performance and Results Act (GPRA) of 1993, and by the NSF’s Strategic Plan. Data collected in the CREST monitoring system help NSF management examine their progress towards the Foundation’s goals and respond to these reporting requirements.

 

 

Finally, the data can also be used as a preliminary step in more detailed future evaluation efforts, such as the sort of rigorous evaluations described in the May 2007 Report of the Academic Competitiveness Council, which was established by the Deficit Reduction Act of 2005 (P.L. 109-171) to serve as a multi-agency effort to identify federal STEM education programs and establish their effectiveness. The full ACC report can be accessed at http://www.ed.gov/about/inits/ed/competitiveness/acc-mathscience/index.html.

 

Data are collected from CREST PIs using the CRESTWeb Data Collection System, an online system developed to facilitate electronic data collection. The three general categories of data that are collected are summarized below:

 

1) Participant information.  Data are collected on the individual participants in the CREST program, who include principal investigators, faculty members, postdoctorates, students, other personnel, and contributors. These data include:

·        Contact information (name, address, phone number, e-mail address, etc.)

·        Demographic data (race, ethnicity, gender, disability, citizenship, etc.)

·         Academic information (academic level, year in school, academic rank, number of students involved, degree, field of study, etc.)

2) Project Implementation Data. Aggregate data are collected on the amount of financial support CREST awardees receive from various sources, including:

·        NSF CREST

·        Other NSF

·        Other Federal agencies

·        State government

·        Local government

·        Industry

·        University

·        Other sources

In addition, data are collected on the CREST involvement of individual participants, including research thrust areas and financial support received.

3) Project accomplishment data.

Aggregate data, including title, year, thrust area, and a narrative description, are collected on each CREST center's: 

·        Proposals (CREST Centers/PIs are actively charged with seeking co-funding. These can include written responses to NSF-issued program solicitations and to other funding sources.)

·        Awards (Funding received as a result of a proposal submitted in response to a program announcement or solicitation.)

·        Presentations (Events such as lectures or speeches that are presented to an audience.)

·        Publications (Any research or technical papers, briefs, articles or other written products that were fostered or funded by CREST during the current reporting period.)

·        Collaborative projects (A CREST initiative undertaken as a joint venture between a CREST-affiliated center and other outside entities. Collaborative projects can also be between two CREST centers.)

·        Patents (Patents awarded to patent applications previously filed with the U.S. Patent and Trademark Office or the European equivalent. In addition, patent applications are tracked, even if the patent application was rejected.)

·         Activities (Educational procedures designed to stimulate learning through firsthand experience.)

A crosswalk of data elements can be found in Appendix B.

A.3. Use of Information Technology To Reduce Burden

Like other tasks under this generic clearance request, CRESTWeb uses advanced electronic information technology to minimize data duplication and respondent burden.  EHR tends to favor Web-based systems because they can facilitate respondents’ data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough editing of all submitted data for completeness, validity, and consistency. Editing is performed as data are entered. Most invalid data cannot enter the system, and questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF. CRESTWeb data fields are marked with out-of-range indicators, warning respondents to check their data if they appear to be out-of-range. Web-based surveys also employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus, and predefined charts and graphics. All these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden. Other efforts to reduce burden include automatically entering each center's contact information and the amount and dates of its award, and allowing users to import previously entered accomplishments to carry them forward to the current round of data collection.

The CRESTWeb system is compliant with Section 508 of the Rehabilitation Act.

A.4. Efforts To Identify Duplication

The CRESTWeb data collection does not duplicate other NSF efforts. Data collected under the CREST program are not available anywhere else, and survey questions are asked only if the information requested is not available elsewhere.. Whenever possible, data are drawn from existing NSF databases, and  NSF has examined its data collection requirements to ensure that the requested data are not available from other Federal sources.

Changes to NSF’s FastLane are monitored on a regular basis, and as new data elements are added to (or deleted from) the FastLane application and/or project reporting system, the survey is modified accordingly.

A.5. Small Business

No information is to be collected from small businesses.

A.6. Consequences of Not Collecting the Information

Without this information, NSF would be restricted in managing and reporting on the activities of awards in the CREST program. Without this feedback, NSF would have no way of making systematic modifications to the CREST program (e.g., adequacy of funding amount, duration of award, and institutional supports needed). These data will ensure that NSF makes informed decisions about future directions of the CREST program. The information requested here is not available elsewhere.

Additionally, without this information NSF would find it difficult to meet agency data requests, as well as GPRA and PART reporting requirements and would be unable to comply fully with congressional and presidential mandates that the Foundation asses its STEM education programs.

A.7. Special Circumstances Justifying Inconsistencies with Guidelines in 5 CFR 1320.6

The data collections will comply with 5 CFR 1320.6.

A.8. Consultation Outside the Agency

The principal investigators (PIs) who will be using the CRESTWeb system were consulted in its design and planning. A pilot test was conducted before the system launched in 2002. Feedback from system users is also solicited at the annual CREST PI meeting, usually held in the spring, and user comments submitted during the collection period are taken into consideration for system improvements.

A.9. Payments or Gifts to Respondents

No payments or gifts will be provided to respondents.

A.10. Assurance of Confidentiality

Respondents will be advised that any information on specific individuals will be maintained in accordance with the Privacy Act of 1974. Data collected are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to Federal and State privacy statues. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c. The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court or party in a court, or Federal administrative proceeding, if the government is a party.

The opening screen on CRESTWeb states the following:

The Federal Government has a continuing commitment to monitor its awards to identify and address any inequities based on gender, race, ethnicity, or disability of the principal investigators, co-principal investigators, trainees, or other participants. Submission of the requested information is not mandatory. If you do not wish to submit the information, please check the boxes provided for this purpose on the Web pages that follow.

Information from this data collection system will be retained by the National Science Foundation, a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). These are confidential files accessible only to appropriate National Science Foundation (NSF) officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested as “for general use,” will be made available to anyone outside of the National Science Foundation for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c.

 

A.11. Questions of a Sensitive Nature

CREST requests data that are considered of a private nature, including the name, phone number, e-mail address, disability status, and citizenship of the PIs, faculty members, and students associated with the activities of a center. In addition, financial support information is collected for faculty members and students. These data are collected in order to monitor the sites' participant populations, to track the participants over time, and to assess the success of the program. Respondents have the option of not providing information that they consider privileged and may mark their gender, race, ethnicity, citizenship, and/or disability as "not reported." Individuals' data are provided only to CREST program staff and consultants conducting studies using the data as authorized by NSF. Any public reporting of the data is in aggregate form.

A.12 Estimates of Response Burden

A.12.1. Number of Respondents, Frequency of Response, and Annual Hour Burden

The estimated average annual response burden is 1,971 hours. This figure was calculated using the total annual burden reported from the last 3 collection cycles. Respondents will be either PIs or program coordinators. One PI or program coordinator per award completes the survey.

 

The estimated annual burden is calculated below.

 

A.12.2. Hour Burden Estimates by Each Form and Aggregate Hour Burdens

The CREST system consists of only one form. This form accounts for the entire annual response burden of 1,971 person-hours.

 

A.12.3. Estimates of Annualized Cost to Respondents for the Hour Burdens

A.13. Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to Respondents or Record Keepers

There is no overall annual cost burden to respondents or record keepers that results from CREST other than the time spent responding to the online survey; the system screen shots are attached as Appendix A.

 

It is usual and customary for individuals involved in implementing a CREST award to keep descriptive records. The information being requested is from records that are maintained as part of normal practice at higher education institutions. Furthermore, the majority of respondents are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, PIs, other center personnel, and students who are the respondents to the CREST data collection task make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance costs under OMB 3145-0058, and free software (e.g., Netscape or Microsoft Explorer) to respond. Thus, there are no capital and startup costs or operation and maintenance costs to respondents or record-keepers.

A.14. Estimates of Costs to the Federal Government

Computing the annualized cost to NSF for the CREST and HBCU-RISE data collection was done by taking the budgets for 3 years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the CREST data collection:

 

 

The annualized cost was computed as one-third of the total 3-year costs; thus, the annualized cost to NSF for the CREST and HBCU-RISE data collection is $216,383.

A.15. Changes in Burden

The previously reported total annual estimated response burden for this collection was 1,022 hours for 14 respondents; the current request for 1,971 hours for 27 respondents is an increase of 949 hours. While the hour burden has remained the same, there is an increase in the number of respondents because data is now being collected from PIs receiving awards under CREST’s HBCU-RISE initiative. This initiative focuses on Historically Black Colleges and Universities and provides awards with the same mission as CREST awards through the same program solicitation (http://www.nsf.gov/pubs/2007/nsf07526/nsf07526.htm). The data collected and forms used by CREST and CREST HBCU-RISE initiative respondents are identical.

There have been no major changes in the instrument that would affect the burden. Minor adjustments to the data elements collected include requesting more complete citation information about publications, additional information on the size at and date of presentations, and collecting data on the funding source of and NSF programs related to collaborative activities.

A.16. Plans for Publication, Analysis, and Schedule

This is a recurring study. Data collection is conducted annually, beginning in early to mid-March and ending in late May. The data are collected for internal review purposes and to monitor the CREST awardees, as well as for use in reporting to Congress and fulfilling NSF’s GPRA and PART reporting requirements. Reports to NSF management and Congress dealing with the characteristics and performance of the CREST program include tables and charts generated from the database. In addition, CRESTWeb users are able to access tables that display summary information for data entered in the current and previous collections.

Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. Macro International Inc., the contractor conducting this data collection on behalf of NSF, is forbidden contractually from publishing results unless NSF has made a specific exception. In short, all products of the collections are the property of NSF. After the products are delivered, NSF determines whether the quality of the products deserves publication verbatim by NSF, and NSF is the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information delivered by the collection that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System ODS or simply a page on the NSF Web site) in which to publish.

Before the conclusion of the collection, both NSF and the funded CREST centers may use preliminary data to improve management and performance. For example, data generated by this study may appear as inputs to other internal and external NSF reports. At this time, NSF has no set timeline for publishing interim reports from this study.

A.17. Approval to Not Display Expiration Date

Not Applicable

A.18 Exceptions to Item 19 of OMB Form 83-I

No exceptions apply.

Section B

Introduction

B.1. Respondent Universe and Sampling Methods

The sample size is the entire universe of projects. There will be an estimated average of 27 awards active each year, and data will be collected from each center. These projects are heterogeneous in terms of their research focus and their academic disciplines in the sciences, technology, engineering, and mathematics.

 

Population	Estimated Universe Size	Sample Size
Awardees	27	27

 

 

B.2. Information Collection Procedures/Limitations of the Study

This data collection uses a Web-based survey. Each respondent will provide answers each year during the duration of their NSF funding and 1 year after NSF-funding has expired.

NSF understands the limitations of the this data collection, particularly in terms of using the data to determine program effectiveness. Data collected through the CREST system are not used to determine the ultimate effectiveness of its STEM educational interventions, but are used in program planning and management, to report on agency activities and goals, and to lay the groundwork for future evaluations.

B.2.1. Statistical Methodology for Stratification and Sample Selection

This collection is a census, so no sampling is required.

B.2.2. Estimation Procedure

Not Applicable

B.2.3. Degree of Accuracy Needed for the Purpose Described in the Justification

Not Applicable

B.2.4. Unusual Problems Requiring Specialized Sampling Procedures

Not Applicable

B.2.5. Use of Periodic (Less Frequent Than Annual) Data Collection Cycles

Not Applicable

B.3. Methods for Maximizing the Response Rate and Addressing Issues of Nonresponse

Past collections have had 100 percent response rates and NSF anticipates that the rate will remain the same. Considerable effort is made for follow up for non-response and incomplete responses. This is done by sending e-mail every three weeks to award sites that have not logged into the system, and by notifying all award sites still entering data when the system closing date is one week away. Approximately 60% of award sites receive at least one of these follow-up e-mail reminders. Examples of the e-mail messages announcing the opening of the system and reminding awardees to log in and enter data are included in Appendix C. The collection is part of reporting required of awardees; principal investigators are responsible for ensuring that data are collected from other center personnel and students, and have access to status information on the Web site indicating which participants have not responded.

B.4. Tests of Procedures or Methods

This system has been operation since 2002 and has been tested extensively. The system is routinely updated, based on user feedback. For example, after receiving feedback from system users, a check was added to verify that duplicate persons and accomplishments are not added to the CRESTWeb system. Also, the Proposal and Award sections were combined into a single section; this streamlined approach facilitates data collection and reduces user burden.

B.5. Names and Telephone Numbers of Individuals Consulted

Agency

Victor Santiago, National Science Foundation, (703) 292-4673

William Neufeld, National Science Foundation, (703) 292-5148

Contractors

Macro International Inc. will be responsible for data collection and analysis under the direction of Lea Mesner, (301) 657-3077.