REQUEST FOR RENEWAL OF CLEARANCE FOR MONITORING FOR THE NATIONAL SCIENCE FOUNDATION'S (NSF's) MATH AND SCIENCE PARTNERSHIP (MSP) PROGRAM
This request for Office of Management and Budget (OMB) review asks for renewal of clearance of the online monitoring system for the Math and Science Partnership (MSP) program, which is funded by the Directorate of Education and Human Resources (EHR) at the National Science Foundation (NSF). The system is already in place, having been granted clearance for the first collection in 2004; this request asks for clearance for the next three (3) years of data collection.
The NSF funds research and education in mathematics, science and engineering. It does this through grants, contracts, and cooperative agreements to more than 2,000 colleges, universities, and other research and/or education institutions in all parts of the United States. The Foundation accounts for about 20 percent of Federal support to academic institutions for basic research. EHR is the directorate within NSF that is responsible for the health and continued vitality of the Nation's science, technology, engineering, and mathematics (STEM) education and for providing leadership in the effort to improve education in these areas.
Over the past four years, NSF has been using results from the online monitoring system to respond, in a timely fashion, to the Congressional mandate to provide ongoing program results on the MSP program. NSF also uses the data to monitor the annual activities and associated outcomes of individual projects. It also contributes substantially to the MSP project and program evaluations and provides important information for NSF's GPRA and PART reports. Based on issues that have emerged from analyses of existing data, we are adding one additional item to the IHE Participant Survey that is designed to examine the extent to which MSP is influencing the disciplinary research of faculty members, as well as an additional item to the K-12 District Survey that is designed to examine whether K-12 schools that met the criteria for significant participation in MSP made Adequate Yearly Progress (AYP) in mathematics for the previous school year.
Timely clearance of this request is critical in order for NSF to continue meeting the Congressional mandate in P. L. 107-368, Sections 9 and 19 to evaluate the MSP program and provide Congress with ongoing results from this evaluation and for NSF to meet its Government Performance and Results Act (GPRA) and OMB's Program Assessment Rating Tool (PART) accountability requirements.
A. Overview of the MSP Program
The MSP program is a major effort under the aegis of President Bush's national education initiative, No Child Left Behind and reauthorized through the America COMPETES Act of 2007 (Public Law 110-69). To date, NSF has made over $610 million in commitments to partnership activities in a research and development portfolio that spans the nation. The goals for the program are to:
Ensure that all K-12 students have access to, are prepared for, and are encouraged to participate and succeed in challenging curricula and advanced mathematics and science courses;
Enhance the quality, quantity, and diversity of the K-12 mathematics and science teacher workforce; and
Develop evidence-based outcomes that contribute to our understanding of how students effectively learn mathematics and science.
Individual projects funded by the MSP program aim to address the aforementioned issues by incorporating a depth and quality of creative strategic actions that extend beyond commonplace approaches. Although all MSP projects share a focus on the same set of fundamental issues, individual MSP projects differ in their scope and are categorized accordingly. The MSP program provides awards to the following types of projects:
Comprehensive Partnerships implement change in mathematics and/or science educational practices in both Institutions of Higher Education (IHEs) and in schools and school districts, resulting in improved student achievement across the K-12 continuum.
Targeted Partnerships focus on improved K-12 student achievement in a narrower grade range or disciplinary focus within mathematics or science.
Institute Partnerships, also referred to as Teacher Institutes for the 21st Century, focus on the development of mathematics and science teachers as school- and district-based intellectual leaders and master teachers.
Research, Evaluation, Technical Assistance (RETA) awards build and enhance large-scale research and evaluation capacity for all MSP awardees and provide them with tools and assistance in the implementation and evaluation of their work.
This clearance request covers a series of existing online surveys that are designed to obtain annual data from principal investigators (PIs) and other program participants for each of these project types.
B. Overview of the Study Design
The MSP Management Information System is designed to collect both quantitative and qualitative data on an annual basis and will allow for comparisons both within and among projects over time. The primary method of gathering this information is by the use of a Web-based data collection system that incorporates the nine surveys covered by this submission (and provided in appendices A through I)—including:
Comprehensive and Targeted MSP Projects
Annual Survey for Comprehensive and Targeted Partnership Projects (Attachment A). This survey collects information on each of the project's partner organizations (e.g., IHEs, K-12 school districts, project evaluators), the grades and subject areas the project will address, project activities by key feature, and involvement with MSP RETA awardees. It can be completed by the principal investigator (PI) or someone designated by the PI. Included in the survey is an Administrative Module that must be completed in order that K-12 district partners, IHE partners, and IHE participants can have access to the system to complete the survey modules for which they are responsible.
Annual IHE Participant Survey for Comprehensive and Targeted MSPs (Attachment B).This survey module collects information about the characteristics and contributions of IHE faculty members and administrators who are active participants in an MSP Comprehensive or Targeted project. Information collected includes demographic characteristics, current fields of research and instruction, and contributions to their MSP. The survey must be completed by each individual IHE faculty member and administrator who is directly supported by the MSP grant and/or directly participated in the development or implementation of MSP-related activities during the previous school year.
Annual Institution of Higher Education Survey (Attachment C). This survey, completed by each MSP IHE partner participating in a Comprehensive or Targeted partnership, obtains information on the number of individuals who developed and/or delivered MSP activities and the number of individuals who were recipients of MSP activities.
Annual K-12 District Survey (Attachment D). This survey, completed by participating K-12 school districts in a Comprehensive or Targeted partnership, collects standardized data about each district and the participating K-12 schools within the district. At the district level, the survey collects information about professional development, K-12 personnel involvement with the development and/or delivery of MSP activities and the number of participating schools within the district. At the school level, the survey collects information about all mathematics and science teachers in the schools with significant MSP involvement, participating mathematics and science teachers, school enrollment, student course enrollment in mathematics and science courses, and student achievement on statewide, criterion-based, mathematics and science accountability assessments.
Institute Projects
Annual Survey for Institute Partnership Projects (Attachment E). This survey collects information on each of the project's partner organizations (e.g., IHEs, K-12 school districts, project evaluators), the scope of the project (e.g., grades and subject areas the project addresses and criteria for selecting teachers) and project activities by key feature. It also collects information about the demographic characteristics of students and teachers in the schools of the K-12 Institute enrollees. It can be completed by the principal investigator (PI) or someone designated by the PI. Included in the survey is an Administrative Module that must be completed in order that IHE participants can have access to the system to complete the survey modules for which they are responsible.
Annual IHE Participant Survey for Institute MSPs (Attachment F). This survey collects information about the characteristics and contributions of IHE faculty members and administrators who are active participants in an Institute MSP project. Information collected includes demographic characteristics, current fields of research and instruction, and contributions to their MSP. The survey must be completed by each individual IHE faculty member and administrator who is participating in an Institute MSP.
Initial Survey for K-12 MSP Institute Participants (Attachment G). This paper-based survey collects information about the characteristics of K-12 teachers and administrators prior to their participation in an Institute MSP project. Information collected includes demographic characteristics, school characteristics, instructional and administrative responsibilities, and educational preparation and certification. The survey must be completed by each K-12 teacher and administrator who is participating in an Institute MSP at the beginning of their participation in the program.
Annual Survey for K-12 MSP Institute Participants (Attachment H). This survey collects information about the characteristics and professional development of K-12 Institute participants. Information collected includes participant’s current professional status, degrees and certifications earned, leadership responsibilities, Institute and professional development activities, and professional community building. This survey is completed annually by each individual K-12 Institute participant who has completed the Initial Survey for K-12 Institute Participant, are active in the MSP Institute program, and whose participation is directly funded by NSF MSP grant.
RETA Projects
Annual Survey for RETA Projects (Attachment I). This survey, completed by the PIs for RETA projects, collects information about each RETA's level of involvement with MSP partnership projects. Annual information is obtained on the numbers of participants, the types of activities that have been conducted (e.g., creating assessment materials or conducting surveys), and who the RETA collaborated with during the previous year.
The MSP program is a major research and development effort that supports innovative partnerships to improve K-12 student achievement in mathematics and science. Cleared in September 2005 for three years as OMB 3145-0199, a renewal of the MSP clearance that allows continued collection of data is requested for a total of nine surveys, listed above. The surveys have not been significantly changed since the last clearance (see Section A.15 for a discussion of two additional items that are being proposed—one of IHE participants and another for the K-12 District Survey). Consistent with the Terms of Clearance, new items will include pre-filled fields were possible in the interest of decreasing respondent burden.
MSP projects are expected to both raise the achievement levels of all students and significantly reduce achievement gaps in the mathematics and science performance of diverse student populations. Successful projects will serve as models that can be widely replicated in educational practice to improve the mathematics and science achievement of all the nation's students.
The MSP program is also directly aligned with all three of NSF's long-term investment categories that:
...link directly to NSF programs and budget resources. They provide the framework for development of more specific and time-dependent performance goals, and for other assessments, such as the PART:
Individuals: Investments that ensure development of world-class scientists, engineers, mathematicians, technologists and educators.
Institutions: Investments that enable colleges, universities and other institutions to attract increased numbers of students to science and engineering (S&E) fields and enhance the quality of S&E education at all levels.
Collaborations: Investments that foster partnerships with colleges, universities, school districts, and other institutions - public, private, state, local, and Federal - to strengthen S&E education at all levels and broaden participation in S&E fields. (p. 14), (http://www.nsf.gov/od/gpra/Strategic_Plan/FY2003-2008.doc)
The MSP program represents an investment in the individual project participants and recipients of MSP activities, the institutions of higher education funded, and the collaborations fostered between K-12 schools and school districts and colleges and universities.
NSF goals and investment categories provide the framework for the development of NSF performance goals. Since the MSP program is a critical part of NSF's efforts to meet these goals, the timely collection of data through the MSP Management Information System is essential for NSF's documentation, as required by the GPRA and PART. In addition, the MSP Management Information System collects standardized information needed to evaluate the success of individual MSP projects and the MSP program as a whole. The 2006 MSP solicitation states that applicants for an MSP award are required to:
"Describe the evaluation plan that will guide project progress annually and will measure the impact of the work described in the action plan, including a description of the instruments/metrics by which partners will document, measure and report on the project’s progress toward realizing improved student and teacher outcomes. The evaluation plan should directly relate to the annual benchmarks and outcome goals in the Appendices section of the proposal. Formative evaluation should provide evidence of the strengths and weaknesses of the project, informing the Partnership's understanding of what works and what does not in order to inform project progress and success. Summative evaluation should give an objective analysis of qualitative and quantitative data, thus demonstrating the effectiveness of the project on student and teacher outcomes and K-20 institutional change. Although the evaluation plan will be developed with input from the Partnership, objective analyses and findings require either an external evaluator or an evaluator within a partner institution who is clearly separate and distinct from the partnership participants and their departments/units (e.g., in a department/unit within a university that is not part of the Partnership itself)." (NSF 06-539, p. 14, http://www.nsf.gov/pubs/2006/nsf06539/nsf06539.htm)
By collecting project-specific information that can be shared with the appropriate evaluators, the MSP Management Information System eliminates the redundancies associated with multiple evaluators developing their own data collection instruments to collect basic information needed for all of the individual project evaluations. At the same time, a single standardized data collection source provides the evaluation of the MSP program with data necessary to determine whether program objectives have been attained and to examine what project characteristics are most closely associated with project success.
The first four cohorts of the MSP program are currently being monitored by the online system. Continuing this data collection activity for these cohorts in future years, as well as monitoring the efforts of future cohorts, is necessary to decipher the extent to which programmatic outcomes are being achieved, as this monitoring system is the only method by which these data are being captured in a consistent manner across all awards.
The primary purpose for this data collection is program planning and management, also known as program monitoring, at the project and program levels. Monitoring the MSP program yields a better understanding of how the program is being implemented and its impact. NSF is using results in responding, in a timely fashion, to the Congressional mandate to provide ongoing program results on the MSP program. NSF also uses the data to monitor the annual activities and associated outcomes of individual projects. It also contributes substantially to the MSP project and program evaluations and provides important information for NSF's GPRA and PART reports. NSF has contracted with Westat, Inc., which developed the monitoring system for the MSP program. All information collected is and will continue to be used to provide analytical and policy support to EHR, assisting NSF to make decisions about future funding and other program initiatives to improve STEM education.
Westat provides NSF with annual reports displaying aggregated data for all MSP projects, as well as project-specific tables for each MSP project. Westat has also made electronic files available to individual MSP projects so they can review and extract their own data to facilitate their management and evaluation tasks. Project-specific data for all projects is available only to EHR staff, EHR contractors with responsibility for impact database management or program evaluators, and the NSF program managers and their staff.
The MSP data collection effort makes maximum use of computer technology to minimize the response burden and to maximize its ability to respond in a timely fashion to Congressionally mandated reporting requirements. Projects use a worldwide Web browser to submit the required data over the Internet using the specially developed MSP online data collection system software. EHR favors Web-based systems because they facilitate respondents' data entry across computer platforms. One feature of the system is the thorough editing of data for completeness, validity, and consistency prior to final submittal. Editing is performed as data are entered. Questionable or incomplete entries are called to respondents' attention before they are submitted to NSF. Features such as automatic tabulations, checkboxes, standard menus, and predefined charts and graphics facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden.
On the Annual IHE Participant Survey for Comprehensive and Targeted MSPs and the Annual IHE Participant Survey for Institute MSPs, certain items are only required of those respondents that participated for 40 or more hours in a given year (i.e., the system is designed to collect only a minimal amount of information from those IHE participants who participated less than 40 hours in their MSP project in any given year). The purpose is to obtain basic information on all IHE participants while minimizing response burden on those individuals who did not meet a specific threshold of participation. Similar procedures are in place in the Annual K-12 Survey to assure that more detailed information is only requested of those schools that have met a specific threshold of participation.
Furthermore, in each data collection period, individual items (e.g., contact information) show respondents' data submitted in earlier years so that these data can be easily updated as opposed to re-entered. In addition, items that will never need to be revised or updated (e.g., type of organization that received the award) are not displayed in subsequent collections. Since most project participation is on a multi-year basis, updating the previous year's data in subsequent collections is far easier and less burdensome than providing the data in the first year.
The online management information system is the only current means of collecting these data (no similar data exist elsewhere). All project data on program funding are drawn from the NSF administrative database now called the FastLane Project Reports system (OMB Control Number 3145-0058). Data collected via the monitoring system will be used, where possible, to pre-fill survey items in subsequent years to further minimize overall response burden.
No information is to be collected from small businesses.
If the information is not collected in a timely fashion, NSF will be unable to document the effectiveness, impacts, or outcomes of the MSP program. It will not be able to meet its Congressionally mandated requirement to evaluate the MSP program and provide Congress with ongoing results from this evaluation and will not meet its accountability requirements or assess the degree to which individual projects are meeting their goals. Moreover, NSF will be unable to comply fully with the Congressional mandate that the Foundation evaluate its STEM education programs.
The data collections will comply with 5 CFR 1320.6.
This data collection was published in Federal Register/Notices March 17, 2008 (Volume 73, Number 52 Page 14276-14277). A copy of the notice is attached in Attachment J.
No payments or gifts will be provided to respondents.
Respondents will be advised that no information on specific individuals will be maintained, in accordance with the Privacy Act of 1974. Data collected are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to Federal and State privacy statutes. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c. The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court or party in a court, or Federal administrative proceeding, if the government is a party.
The types of questions asked on these surveys are not considered sensitive. Furthermore, every effort has been made to protect the privacy of individuals involved in the MSP program. The only individually identifiable information collected by the surveys is the name and contact information for persons completing the surveys and/or supplying the data reported. This information is needed to allow Westat staff and project evaluators to follow-up with any necessary clarifying questions. With these exceptions, the system has been designed so that neither Westat nor NSF will have information permitting them to identify MSP participants. Participants are listed in the online system using an identification number. The ID number is maintained by each MSP project, along with the individuals' contact information. This information is used to track recipients of funding and training. Although the two Annual IHE Participant Surveys do ask for some demographic information, neither Westat nor NSF has the capability to link the information to any individual. Any individualized data that are collected are provided only to program staff, consultants, and contractors conducting studies using the data as authorized by NSF. Any public reporting of these data is in aggregate form.
As mentioned above, the MSP Management Information System is designed to collect both quantitative and qualitative data on an annual basis that will allow for comparisons both within and among NSF-funded projects over time. All but one of the nine surveys is administered via an online system. Four of these surveys collect data from individuals (i.e., Annual IHE Participant Survey for Comprehensive and Targeted MSPs, Annual IHE Participant Survey for Institute MSPs, Initial Survey for K-12 MSP Institute Participants, and Annual Survey for K-12 MSP Institute Participants) and are designed to obtain basic information about individual participants and their participation in MSP activities. In keeping with the NSF's MSP program monitoring goals, all nine instruments are designed to collect data that are easily accessible to respondents.
We estimate that the total number of annual respondents will be 3,149. The estimated annual response burden is 50,322.
The number of respondents per survey instrument and burden hour calculations are presented in Chart 1. Burden hours per response are estimated on the basis of discussions with NSF, PIs, and Westat's experience in administrating the surveys over the previous four years. Assumptions for the number of respondents are based on the number of respondents for the 2005-06 collection cycle (the most recent collection cycle for which information is currently available). Information about changes between the current estimates and the estimates in our last request for MSP survey clearance is contained in Section A.15.
Chart 1 Annual Burden Hours for MSP Monitoring System, by Type of Respondent
|
Number of Respondents |
Burden Hours per Respondent |
Annual Person-Hour Total* |
Comprehensive and Targeted Projects |
|||
Annual Survey for Comprehensive and Targeted Partnership Projects - Principal Investigator |
39 |
55 |
2,145 |
Annual Targeted and Comprehensive MSP IHE Participant Survey |
1,250 |
0.83 |
1,038 |
Annual IHE Survey for Comprehensive and Targeted MSPs -- IHE Partners |
134 |
8 |
1,072 |
Annual K-12 District Survey |
692 |
64 |
44,288 |
Institute Projects |
|||
Annual Survey for Institute Partnership Projects - Principal Investigator |
12 |
60 |
720 |
Annual IHE Participant Survey for Institute MSPs |
150 |
0.83 |
125 |
Initial Survey for K-12 MSP Institute Participants |
200 |
0.25 |
50 |
Annual Survey for MSP Institute K-12 Participants |
660 |
1.33 |
878 |
RETA Projects |
|||
Annual Survey for RETA Projects |
12 |
0.5 |
6 |
Total respondents |
3,149 |
|
50,322 |
* = Number of Respondents x Burden Hours per Response
There are a total of nine survey forms that we are requesting clearance for use in this study: 1) the Annual Survey for Comprehensive and Targeted Partnership Projects, 2) the Annual IHE Participant Survey for Comprehensive and Targeted MSPs, 3) the Annual Institution of Higher Education Survey 4) the Annual K-12 District Survey, 5) the Annual Survey for Institute Partnership Projects, 6) the Annual IHE Participant Survey for Institute MSPs, and 7) the Initial Survey for K-12 MSP Institute Participants, 8) the Annual Survey for MSP Institute K-12 Participants, and 9) the Annual Survey for RETA Projects. Eight are Web-based surveys and one (the Initial Survey for K-12 MSP Institute Participants) is paper-based.
The hour burden estimates by type of form are presented in Chart 2:
Chart 2. OMB Burden Calculation for MSP Monitoring System, by Survey Form
|
Number of Respondents |
Number of Responses Per Respondent (over 3 years) |
Burden Hours per Response |
Total Hour Burden (over 3 years)* |
Annual Average Hour Burden** |
Comprehensive and Targeted Projects |
|||||
Annual Survey for Comprehensive and Targeted Partnership Projects - Principal Investigator |
39 |
3 |
55 |
6,435 |
2,145 |
Annual Targeted and Comprehensive MSP IHE Participant Survey |
1,250 |
3 |
0.83 |
3,113 |
1,038 |
Annual IHE Survey for Comprehensive and Targeted MSPs -- IHE Partners |
134 |
3 |
8 |
3,216 |
1,072 |
Annual K-12 District Survey |
692 |
3 |
64 |
132,864 |
44,288 |
Institute Projects |
|||||
Annual Survey for Institute Partnership Projects - Principal Investigator |
12 |
3 |
60 |
2,160 |
720 |
Annual IHE Participant Survey for Institute MSPs |
150 |
3 |
0.83 |
374 |
125 |
Initial Survey for K-12 MSP Institute Participants |
200 |
1 |
0.25 |
50 |
50 |
Annual Survey for MSP Institute K-12 Participants |
660 |
3 |
1.33 |
2,633 |
878 |
RETA Projects |
|||||
Annual Survey for RETA Projects |
12 |
3 |
0.5 |
18 |
6 |
Total respondents |
3,149 |
|
|
150,863 |
50,322 |
* = Number of Respondents x Number of Responses Per Respondent x Burden Hours per Response
** = Total Hour Burden (over 3 years) / 3
The overall annual costs to respondents for burden hours are estimated to be $2,227,582. The hourly wage rates were based on information found in the Department of Education's National Center for Educational Statistics Integrated Postsecondary Education Data System (Table 3, Employees in Postsecondary Institutions, Fall 2006, and Salaries of Full-Time Instructional Faculty, 2006-07, http://nces.ed.gov/pubs2008/2008172.pdf) and the U.S. Department of Labor, Bureau of Labor Statistics, National Compensation Survey (Table 2, National Compensation Survey: Occupational Wages in the United States, June 2006, June 2007, Summary 07-03), adjusted for inflation.
Calculations are shown in Chart 3.
Chart 3. Annualized Cost to Respondents, by Type of Respondent
|
Annual Average Hour Burden |
Estimated Hourly Wage Rate |
Estimated Cost to Respondents* |
Comprehensive and Targeted Projects |
|||
Annual Survey for Comprehensive and Targeted Partnership Projects - Principal Investigator |
2,145 |
$47.00 |
$100,815 |
Annual Targeted and Comprehensive MSP IHE Participant Survey |
1,038 |
$45.34 |
$47,063 |
Annual IHE Survey for Comprehensive and Targeted MSPs -- IHE Partners |
1,072 |
$45.34 |
$48,604 |
Annual K-12 District Survey |
44,288 |
$44.27 |
$1,960,630 |
Institute Projects |
|||
Annual Survey for Institute Partnership Projects - Principal Investigator |
720 |
$47.00 |
$33,840 |
Annual IHE Participant Survey for Institute MSPs |
125 |
$45.34 |
$5,668 |
Initial Survey for K-12 MSP Institute Participants |
50 |
$33.06 |
$1,653 |
Annual Survey for MSP Institute K-12 Participants |
878 |
$33.06 |
$29,027 |
RETA Projects |
|||
Annual Survey for RETA Projects |
6 |
$47.00 |
$282 |
Total respondents |
50,322 |
|
$2,227,582 |
* Annual Average Hour Burden x Estimated Hourly Wage Rate
There is no overall annual cost burden to respondents or record keepers that results from the MSP program other than the time spent responding to the surveys that are attached as appendices to this request.
It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, the PIs, school districts, IHE partners and IHE participants who are the primary respondents to the individual MSP data collections tasks make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance cost under OMB 3145-0058, and free software (e.g., Netscape or Microsoft Explorer) to respond.
The total estimated cost to the government of all data collection, analysis, and reporting activities for this study is approximately $1,260,000. This estimate is based on the actual annual cost of the collection in 2007 (shown in Chart 4).
Chart 4. Estimated Annual Cost to the Federal Government of Collection (based on 2007 expenditures)
Personnel |
$147,113 |
Travel |
$38 |
Computing |
$10,712 |
Copying |
$930 |
Postage |
$149 |
Overhead |
$158,221 |
G&A and Fee |
$101,487 |
Total Costs |
$418,651 |
Based on issues that have emerged from analyses of existing data, we are adding one additional item to the IHE Participant Survey that is designed to examine the extent to which MSP is influencing the disciplinary research of faculty members. We anticipate that this new item will add an average of five (5) minutes to the time required to complete the IHE Participant Survey.
We are also adding one additional item to the K-12 District Survey that is designed to examine whether K-12 schools that met the criteria for significant participation in MSP made Adequate Yearly Progress (AYP) in mathematics for the previous school year. The addition of this item will enable us to augment our assessment of whether schools that are participating in MSP are meeting one of the primary educational measures that is currently used to assess the mathematical attainment of K-12 students. Projects will be able to obtain this information from websites maintained by the Department of Education for their state. We anticipate that this new item will add an average of two (2) hours to the time required for projects to complete the K-12 District Survey (the time required to complete this item will vary depending on the number of schools in a given project that have met the criteria).
Chart 5 summarizes the changes in survey items between the 2005 request for clearance and the current request.
Chart 5. Major Changes in Survey Items Since Last OMB Clearance Request
Attachment |
Collection title |
Changes since 2005 Request |
|
Status |
Change in burden per project |
||
A |
Annual Survey for Comprehensive and Targeted Partnership Projects |
No Change |
n/a |
B |
Annual IHE Participant Survey for Comprehensive and Targeted MSPs |
Added a single narrative item that obtains information on how participation in MSP has affected respondents’ disciplinary research |
Additional 5 minutes per respondent |
C |
Annual Institution of Higher Education Survey |
No Change |
n/a |
D |
Annual K-12 District Survey |
Add a single item about whether schools that met the criteria for significant participation made Adequate Yearly Progress |
Additional 2 hours per project |
E |
Annual Survey for Institute Partnership Projects |
No Change |
n/a |
F |
Annual IHE Participant Survey for Institute MSPs |
Add a single item about whether schools that met the criteria for significant participation made Adequate Yearly Progress |
Additional 5 minutes per respondent |
G |
Initial Survey for K-12 MSP Institute Participants |
No Change |
n/a |
H |
Annual Survey for MSP Institute K-12 Participants |
No Change |
n/a |
I |
Annual Survey for RETA Projects |
No Change |
n/a |
Chart 6 summarizes changes in annual hour burden for the system of surveys between what was requested in 2005 and what is currently requested.
Chart 6. Hour Changes in Task Burdens
Attachment |
Collection title |
Annual Burden Requested in 2005 |
Currently Requested Annual Burden |
Change in Burden |
A |
Annual Survey for Comprehensive and Targeted Partnership Projects |
1,680 |
2,145 |
465 |
B |
Annual IHE Participant Survey for Comprehensive and Targeted MSPs |
723 |
1,038 |
315 |
C |
Annual Institution of Higher Education Survey |
1,088 |
1,072 |
-16 |
D |
Annual K-12 District Survey |
30,256 |
44,288 |
14,032 |
E |
Annual Survey for Institute Partnership Projects |
480 |
720 |
240 |
F |
Annual IHE Participant Survey for Institute MSPs |
145 |
125 |
-20 |
G |
Initial Survey for Institute K-12 Participants |
144 |
50 |
-94 |
H |
Annual Survey for Institute K-12 Participants |
765 |
878 |
113 |
I |
Annual Survey for RETA Projects |
10 |
6 |
-4 |
|
Total |
35,291 |
50,322 |
15,031 |
Some of the changes in hour burden are attributable to a change in our estimates of the number of respondents, as detailed in Chart 7. For example, the number of individuals completing the Initial Survey for K-12 MSP Institute Participants (Attachment G) decreased significantly, as some Institute MSP Projects are no longer accepting as many new participants in a given year.
Chart 7. Changes in Number of Respondents
Attachment |
Collection title |
Number of Respondents in 2005 Request |
Respondents in Current Request |
Change in Number of Respondents |
A |
Annual Survey for Comprehensive and Targeted Partnership Projects |
40 |
39 |
-1 |
B |
Annual IHE Participant Survey for Comprehensive and Targeted MSPs |
964 |
1,250 |
286 |
C |
Annual Institution of Higher Education Survey |
136 |
134 |
-2 |
D |
Annual K-12 District Survey |
488 |
692 |
204 |
E |
Annual Survey for Institute Partnership Projects |
8 |
12 |
4 |
F |
Annual IHE Participant Survey for Institute MSPs |
193 |
150 |
-43 |
G |
Initial Survey for K-12 MSP Institute Participants |
575 |
200 |
-375 |
H |
Annual Survey for MSP Institute K-12 Participants |
575 |
660 |
85 |
J |
Annual Survey for RETA Projects |
19 |
12 |
-7 |
|
Total |
2,998 |
3,149 |
151 |
Chart 8 summarizes the timeline for the data collections and annual contractor reports for the first three data collection efforts.
Chart 8. MSP Work plan
Data Collection Tasks |
Date to start |
Date to complete |
Adaptation and revision of current data collection software to incorporate additional items |
5/2008 |
6/2008 |
Software testing and revision |
6/2008 |
7/2008 |
Submission of clearance package to OMB |
|
7/2005 |
Receipt of OMB clearance (assuming received on day 60) |
|
8/2005 |
Web data collection |
8/2008 |
11/2008 |
Follow-up Phone Calls to PIs |
9/2008 |
11/2008 |
Final Cleaning and Validation of Databases |
11/2008 |
4/2009 |
Tabulations for Report |
5/2009 |
6/2009 |
Draft Report to NSF |
|
7/2009 |
Project Reports to NSF |
|
9/2009 |
Files for Program Evaluation |
|
9/2009 |
Additional Reports |
TBD |
|
Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. Westat is conducting this third-party study of the MSP program on behalf of NSF and is forbidden contractually from publishing results unless NSF has made a specific exception. In short, all products of the collections are the property of NSF. After the products are delivered, NSF determines whether the quality of the products deserves publication verbatim by NSF, i.e., NSF is the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information delivered by the study that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System (ODS) or simply a page on the NSF Web site) in which to publish.
Before the conclusion of the study, both NSF and the funded MSP projects may use preliminary data to improve management and performance. For example, data generated by this study are expected to be inputs to other internal and external NSF reports (e.g., the GPRA Annual Performance Plan). At this time, NSF has no set timeline for publishing interim reports from this study.
Not Applicable
Not Applicable
| File Type | application/msword |
| File Title | Supporting Statement |
| Author | Gavin Fulmer |
| Last Modified By | nsfuser |
| File Modified | 2008-06-12 |
| File Created | 2008-06-12 |