Request for Clearance
NASA Explorer Schools (NES) Pilot Implementation Small Scale Evaluation Research Study
Section A
Introduction
A report by the National Research Council (NRC) in 2007 titled, “NASA’s Elementary and Secondary Evaluation Program: Review and Critique,” recommended that the NASA Explorer Schools (NES) model be redesigned. The report states that the original NES model was too ambitious in scope and used too many resources in too few schools. The report recommended that the new NES model reach more schools and students, and focus on motivating students around NASA themes so that they are exposed to and consider STEM careers.
The new NASA Explorer Schools (NES) model will focus implementing a “best of” collection of NASA content and curricular support resources. Eight to ten core products or opportunities will be available, representing all NASA Mission Directorates and current NASA missions in Earth and Space Science, mathematics, chemistry, and physics. Resources will be updated annually, and will include curriculum support guides, design challenges, problem based learning sets, mission-based educational support materials, lesson plans, multimedia, and hands-on engagement opportunities. A NES Virtual Campus will provide short duration professional development experiences for educators. In addition to training on specific STEM topics, the Virtual Campus will provide social networking interactions between peers, and with NASA educators and subject matter experts. Special tools and resources will enable students and educators to make logical connections between areas of study and real-life applications. Tools will include video clips of “teachable moments” and design challenges.
As part of redesigning the NASA Explorer School (NES), NASA's Education Coordinating Committee (ECC), a collaborative structure that maximizes the agency's ability to maintain an integrated education portfolio and strategically manage the implementation of numerous programs, recommended that the NES redesign include a pilot implementation with a small scale evaluation research study which will produce a report to aid with planning and managing the full scale implementation of the NES project. The NES project will be piloted from March 2010 to June 2010. This small scale evaluation research study will produce a formative report on “key components” of the NES project for NES project management to aid with the full scale implementation of NES in Fall 2010. The report will be made available to NES project management, NASA OE senior leadership and NES national partners in July 2010.
A.1 Circumstances Requiring the Collection of Data
The National Aeronautics and Space Administration (NASA)
Office of Education (OE) seeks clearance to administer brief surveys, conduct telephone interviews, and focus groups with thirty teachers, and administer a brief survey to the teacher’s students to conduct a small scale evaluation research study during the three month pilot implementation of the redesigned NASA Explorer Schools (NES). The Space Act 1958 ---- The National Aeronautics and Space Act http://www.nasa.gov/offices/ogc/about/space_act1.html#NASA established NASA.
NASA Office of Education manages several national NASA education projects including NES. The purpose of conducting the pilot implementation of the NASA Explorer Schools and conducting a small scale evaluation research study is to produce a report to provide information for planning and management purposes for full scale implementation of NES in fall 2010.
A.2. Purposes and Uses of the Data
Data collected by survey, telephone interview, and focus group from teachers and data collected by survey from students will be used to produce a report for NASA Education leadership for planning and management of full scale implementation of the NES project. Mix methods of data collection will obtain information and advice from 30 lead teachers and their students in the 30 schools in which the NES pilot will be implemented to improve the full-scale implementation of the NES project.
A majority of data collection will focus on the implementation of NES STEM teaching resource materials, teacher professional development, and on-line media for classroom use. Whenever possible, data sources will be triangulated to tell a more complete and detailed story of NES implementation.
The primary source of data collection will be the Lead Teacher at each NES pilot school, since, the Lead Teacher will be the person at each school most involved in the implementation of NES. Lead teachers will be required to complete all data collection.
Following are descriptions of data collection instruments used in the evaluation research study of the NES pilot.
Student Pre-Pilot/Post-Pilot Survey – Students of the thirty Lead Teachers implementing NES will be requested to complete a brief survey prior to the pilot and at the end of the pilot. Questions on the survey will be very similar to the Office of Education Performance Measurement (OEPM) Student Long-Engagement Survey. The survey will have about 16 items that ask students how likely they will pursue a career with NASA or in a STEM field in the future.
Teacher Professional Development Survey – At the end of each professional develop session, teachers will be required to complete a brief electronic survey (5 to 6 items) which will ask how useful the professional development was, and how likely the teacher will use STEM teaching resource materials and on-line NES media after the professional development. The NES pilot is using three types of professional development (PD): 1) instructor lead PD, 2) Self-guided PD, 3) Facilitated and Self-guided PD. We want to understand which type of PD is preferred and if teachers were more likely to use NASA materials after a particular type of PD. The self-guided PD is the least expensive, but, is it better for NES to invest in instructor lead PD?
NES Lead Teacher Survey – Lead teachers will be surveyed mid-way through the implementation of the NES pilot and at the end of the pilot. Questions on the survey will ask which types of STEM teaching resource materials and on-line media they have used and what did they think of them.
NES Lead Teacher Telephone Interview Protocol – Lead teachers will be asked questions by telephone at the end of the pilot. Questions will focus on the three main components of the NES pilot: NES STEM teaching resource materials, teacher professional development, and on-line media for classroom use. Teachers will be asked their opinion on the usefulness of each, any problems/barriers with using each, and suggestions for improvement.
NES Lead Teacher Focus Groups – At the end of the NES pilot, teachers will travel to a NASA center for focus groups. Three focus groups will be held in the following areas: NES STEM teaching resource materials, teacher professional development, and on-line media for classroom use. Teachers will be asked what they thought of each, and how each could be improved.
A.3 Use of Information Technology to Reduce Burden
NASA surveys have user-friendly features (e.g. custom controls such as check boxes.) It complies with Section 508, the 1998 amendment to the Federal Rehabilitation Act, which mandates that the electronic and information technology used by Federal agencies be made accessible to all people with disabilities.
A.4 Efforts to Identify Duplication
This information to be supplied des not duplication on any other information collection. Since NES is a newly re-designed project, no data on the new NES project currently exists. As mentioned in the response to item A.1. the Circumstances Requiring the Collection of Data, the purpose of this collection is to conduct a small scale evaluation research study during the three month pilot implementation of the redesigned NASA Explorer Schools (NES) to produce a report to provide information for planning and management purposes for full scale implementation of NES in fall 2010.
A.5 Small Business
Not Applicable
A.6 Consequences of Not Collecting the Information
If the data collection is not conducted, NASA will not be able to conduct a small scale evaluation research study during the three month pilot implementation of the redesigned NASA Explorer Schools (NES), nor will not be able to produce a report to provide information for planning and management purposes for full scale implementation of NES in fall 2010. NASA would lack information for making well informed planning and management decisions related to full-scale implementation of the NES project.
A.7 Special Circumstances Justifying Inconsistencies with Guidelines in 5 CFR 1320.6
All data will be collected in a manner consistent with the guidelines of 5 CFR 1320.6.
A.8. Consultation Outside the Agency
NASA Education leadership is consulting with national educational organization to help shape the evaluation research study to keep data collection burden to teachers a minimum and to keep data collection relevant. These partners have met at NASA Headquarters to review and discuss the evaluation in late November ’09 and mid-January ‘10. The partners include: the International Center for Leadership in Education (ICLE), International Technology Education Association (ITEA), and the National Science Teachers Association (NSTA). The national partners are helping with the recruitment of 30 schools for this pilot implementation.
A.9. Payments or Gifts to Respondents
Not Applicable
A.10. Assurance of Confidentiality
Any required assurances of confidentiality will be provided in writing at the top of each survey and in the introduction of the telephone interview protocol, and the focus group protocol.
Teachers will not be assured confidentiality. Student’s names/information will be protected. Aggregate data will be provided to NASA with no identifying information.
A.11. Questions of a Sensitive Nature
Not applicable
12. Estimates of Response Burden.
Total
Hours: 200 hours (see Information Collection Burden table for more
additional details). The Lead Teacher (1 teacher) from 30 schools (a
total of 30 teachers) will complete a survey half-way through pilot
implementation and at the end of the pilot. The survey is
anticipated to take up to 0.5 hours for each time the survey is
completed for a total of 30 hours. At the end of implementation, 1
teacher from 30 schools (a total of 30 teachers) will be expected a
telephone interview that is expected to take .5 hours for a total of
15 hours. Each of the 30 teachers will be expected to participate in
3 focus groups. Each focus group will take 1 hour for a total of 90
hours. Each of the 30 teachers will be required to complete a brief
5 minute survey after each professional development activity.
Teachers are expected to complete six professional development
activities during the pilot for a total of 15 hours. Students of the
30 teachers are required to complete a 15 minute pre-pilot and
post-pilot survey. 30 teachers X 20 students per teacher X 2 times X
.25 hours = 50 hours. The total burden on the public for this data
collection is 200 hours.
A.12.1 Number of Respondents, Frequency of Response, and Annual Hour Burden
|
Customer |
Collection Instruments |
Frequency |
Estimated Number of Respondents |
Expected Response Rate |
Completion Time (Minutes) |
Total Estimated Burden Hours |
1 |
Lead Teachers who participate in the pilot of the NASA Explorer School (NES) |
Survey – NES Lead Teacher Survey
|
One time (April 2010) |
30 Participants |
100% |
30 minutes (.5 hours) |
15 hours |
2 |
Lead Teachers who participate in the pilot of the NASA Explorer School (NES) |
Telephone interview
|
One-time. (June 2010) |
30 Participants |
100% |
30 minutes (.5 hours) |
15 hours |
3 |
Lead Teachers who participate in the pilot of the NASA Explorer School (NES) |
Focus Groups |
4 Focus Groups (June 2010) |
30 Participants/7 to 8 participants per focus groups |
100% |
1 hour for each focus group |
90 hours |
4 |
Lead Teachers who participate in the pilot of the NASA Explorer School (NES) |
Survey – Teacher Professional Development Survey |
Completed each time a teacher draws on a NES resource (estimated 6 times per teacher) |
30 Participants |
100% |
5 minutes (.08 hours) per survey
|
15 hours |
5 |
Students of Lead Teachers who participate in the pilot of the NASA Explorer School (NES) |
Survey – Student Pre-Pilot/Post-Pilot Survey |
Two times (pre-pilot/post-pilot) |
30 teachers X 120 students per teacher = 3600 students |
100% |
15 minutes (0.25 hours) to complete each survey |
3600 student X 15 minutes (.25 hours) X 2 (per/post pilot) = 1800 hours |
|
|
|
Total Number of Respondents |
720 Participants |
|
Total Hours |
1935 Hours |
A.12.2 Hour Burden Estimates by Each Form and Aggregate Hour Burdens
There is no cost to respondents other than the time it takes to respond to the survey.
The hourly rate for educators was established by using the Bureau of Labor Statistic's May 2006 National Occupational Employment and Wage Estimates, which estimates the mean annual wage for those in Education, Training, and Library Occupations to be $45,320. This average annual wages was then divided by the number of standard annual work hours (2,080) to determine an average hourly rate. The rates and the total costs are indicated in the table below:
Educator |
(135 hours x $22/hour) |
$2,970 |
A,13 Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to Respondents or Record Keepers
Not Applicable
A.14. Estimates of Costs to the Federal Government
This small scale evaluation research study will occur one time. There are no annualized costs to the Federal government.
A.15 Change in Burden
New collection
A.16 Plans for Publication, Analysis, and Schedule
Like many agencies, NASA has reduced reliance on traditional, paper-based publication methods and publication formats. Any contractor who assists in gathering the data is forbidden contractually from publishing results unless NASA has made a specific exception. In short, all products of these collections are the property of NASA. After the data are delivered, NASA determines whether the quality of the data deserves publication by NASA, i.e., NASA is the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information that NASA can make a decision on the format (raw or analytical) and manner in which to publish.
The requested information will be used in a NASA Office of Education internal report for planning and management purposes for the implementation of the NES project at scale. Senior Leadership in NASA Office of Education at NASA Headquarters and the NES project manager will use the report.
A.17 Approval to Not Display Expiration Date
Not Applicable
A.18 Exceptions to Item 19 of OMB Form 83-I
Not Applicable
The universe for this census data collection is approximately 30 educators. We plan for one teacher from each school to participate in the NES pilot. Since teachers are required to complete all surveys, telephone interviews, and participate in focus groups in order for a school to be accepted into the NES pilot, we expect all teachers will complete all data collection. A contractor will follow-up with each teacher via email and/or telephone until the teacher completes all required data collection.
Students
The universe of students for this data collection is all students in classes in which a teacher participating in the NES pilot uses a NES module. Teachers will be asked to use distribute the students pre/post surveys to students to all classes in which they use an NES module. We anticipate a census of students. A contractor will follow-up with each teacher via email and/or telephone until each teacher has submitted all pre/post student surveys.
B.2 Information Collection Procedures/ Limitations of the Study
The data collection uses web-based surveys, telephone interviews and in person focus groups. Lead teachers complete a teacher surveys, professional development surveys, telephone interviews and in person focus groups. Students of lead teachers complete a pre and post pilot survey. The census data collection methods will collect data from the lead teacher at each NES pilot school and data from all of their students. Since response to surveys, interviews and focus groups is required for participation in the NES pilot, a response rate of 100% is expected.
Not applicable.
Not applicable.
Not applicable.
Not applicable.
Lead Teachers must agree to complete all data collection activities before their school can participant in the NASA Explorer School pilot.
Not applicable
Agency Unit
Brian Yoder, Evaluation Manager, Office of Education, NASA HQ 202-358-7338.
Mary Sladek, Outcome Manager, Informal Education, NASA HQ 202-358-0861
Rob LaSalvia, NASA Explorer Schools Project Manager, NASA Glenn Research Center, 216-433-8981
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | gaddisd |
| File Modified | 0000-00-00 |
| File Created | 2021-02-02 |