A
Circumstances Making Information Collection Necessary
GLOBE is an ambitious science and education program that continues to evolve even as it penetrates thousands of classrooms in the United States and around the world. The University Corporation for Atmospheric Research (UCAR) took over the management of the GLOBE Program in 2003, and worked closely with SRI to move into the next phase of program evaluation. Currently, GLOBE materials development and professional development training are being organized in a phased approach, with a new "E-guide" that was completed in 2005 and new protocol development and associated activities and learning modules under development in 2006-2008 in response to the NASA/NSF MOU that resulted in 4 new Earth System Science Projects (ESSPs) for the GLOBE Program. Individuals responsible for developing and/or revising these materials and for managing the new NSF ESSPs need to know how GLOBE Partners are facilitating teachers to use GLOBE materials, which parts of the program are working well, and which parts require improvement. In addition, the program needs basic information concerning how many students and teachers at particular grade levels and settings are participating in the program. Program managers need to understand the number of students and teachers being affected by GLOBE and the impact that the program is having on teaching and learning, to make optimal resource management decisions.
How, and by Whom, and for What Purpose Information Is to Be Used
SRI conducted program evaluation for the first 10 years of the program and set the stage for future program evaluation in an organized fashion (1995-2005). The proposed 2006-08 study will allow GLOBE to both measure whether GLOBE is effective and the conditions under which it is effective as a means of achieving self-sustainability in the near future through a comprehensive annual evaluation. The Annual Partner Survey collects, initiated in 2005, gathers information regarding partnership funding and sustainability, programmatic implementation and capacity building, trainers and trainer support, teachers and teacher support, communication and administrative support and education and environmental issues relevant to their state and/or country. The GLOBE Annual Evaluation reports on multiple constituencies, each of whom is likely to use the evaluation for their own purposes as shown in Table 2.
T
Likely Application of Evaluation Findings |
Constituency
|
Purpose of Application |
Build a framework that promotes collaboration between all members of the GLOBE community and facilitates capacity building and self-sustainability |
GLOBE Program Office |
To assist Partners to build strong models of regionalization to build infrastructure and promote future sustainability of the program around the world. |
Adjustment of train-the- trainer offerings and partner support |
GLOBE’s International and U.S. Partners |
To refine the GLOBE program and improve partnerships' effectiveness; promote capacity building. |
Adjust training and post- training support offerings to teachers |
GLOBE's International and U.S. Partners |
To adjust offerings to reflect findings about most effective services partners can offer to teachers. |
Design new approaches to structuring collaborative relationships with schools |
GLOBE Community: GLOBE Science PIs, Partners, Teachers, Students; GLOBE Program Office |
To improve data collection and reporting and the application of both to student research that leads to advances in science. |
Align GLOBE with standards and policies to make it easier for teachers to adopt inquiry‑ based science curricula |
District- and state-level administrators |
To facilitate GLOBE implementation in the U.S. and abroad. |
Share new strategies for implementing GLOBE effectively in classrooms |
GLOBE Community: GLOBE Science PIs, Partners, Teachers, Students; GLOBE Program Office |
To improve student achievement within GLOBE. |
Online Evaluation needs; ease of data entry and mechanisms supporting student research |
GLOBE Community: GLOBE Science PIs, Partners, Teachers, Students; GLOBE Program Office |
To improve the functionality of the GLOBE Web site. |
Use of Improved Information Technology to Reduce Burden
The GLOBE Partner survey is an online instrument developed in 2005 and housed within the GLOBE Partner Administrative section of the GLOBE Web site. This survey is administered in January of each year. The survey gathers data on all activities related to GLOBE implementation for the year prior to administration of the survey. Therefore, the 2005 GLOBE Partner Survey was administered in January 2006 and the 2006 GLOBE Partner Survey will be administered in January 2007, gathering information for all GLOBE community activities that occurred during 2006.
Efforts to Identify Duplication
SRI no longer serves as the
external evaluator of The GLOBE Program. At this time, the GLOBE
Program Office is the only entity conducting surveys, interviews, and
assessments of the impact of GLOBE world-wide. All Annual GLOBE
Surveys distributed to the GLOBE community, including the optional
online surveys regarding Web site redesign, school participation in
the new NSF ESSP projects and evaluation of GLOBE events, are
directed by the GLOBE Program Office (GPO).
Impact on Small Entities
No small business entities are proposed as respondents. All possible efforts have been made to prepare and carry out the surveys, interviews, and assessments in an online format as well as through the GLOBE Partners and Teachers so as to minimize the burden on GLOBE students and their schools, which are mostly entities associated with local governing bodies.
Consequence If Collection Conducted Less Frequently
The GLOBE Program has a built-in flexibility that enables it to make necessary changes as it develops. These changes will be made on the basis of reliable information that is gathered from the GLOBE community whose constituents include Country Coordinators, U.S. Partners, Science PIs, Teachers and their Students (partners serving as the voice for the teachers and students that they work with). This is one of the main purposes of the evaluation.
There is an ongoing need of the program to understand the impact of GLOBE on student learning, because so many new students are exposed to the program each year. In the past, GLOBE has either measured program effectiveness using a global measure of implementation (e.g., data reporting), investigated aspects of implementation (e.g., data analysis, protocol implementation) and employed SRI to conduct teacher/student surveys.
The change to one (1) required Annual Evaluation for all GLOBE Partners ensures that the GLOBE Program Office will be provided with timely information that adequately represents the international nature of the program which is implemented in over 100 countries world-wide.
C
Data collection is fully in compliance with OMB data collection guidelines in 5 CFR 1320.6 (Controlling Paperwork Burdens on the Public-General Information Collection Guidelines).
Consultation with Persons outside the Agency
A number of professional researchers outside the federal agencies that have been involved in GLOBE have been consulted in the design of the current online evaluation and in the development of surveys, interview protocols, and assessments used by the GLOBE Program. They include:
Evaluation Project Development by SRI. Dr. William Penuel, Senior Education Researcher, served as co-PI on the GLOBE evaluation project ending in 2005 and has led a number of large-scale evaluations of educational technology programs, including the Department of Education's Community Technology Centers Program. He also has experience in developing assessments of science inquiry, and is PI for an NSF-funded project focused on the development of formative assessment tools to improve project-based science. Dr. Penuel and his team from SRI were involved in all aspects of the development of the GLOBE Annual Survey currently in use.
Payment of Gifts to Respondents
No payment of gifts is provided to respondents of GLOBE Program evaluations.
Assurance of Confidentiality
Respondents work exclusively inside the GLOBE Program Administrative pages of the GLOBE Web site when responding to the GLOBE Annual Evaluation. Each GLOBE Partner records their programmatic information in their secured space on the Web site and each Partner has the ability to print out a report of their survey responses for their own use. All Partners have been provided with information explaining the purpose for data collection and a summary document of all information obtained through the survey each year is provided to the entire GLOBE community at each GLOBE Annual Conference.
Justification
for Questions of Sensitive Nature
No
items of a sensitive nature are contained in these instruments.
Estimates of Respondent Burden
The
respondent burden for the recipients of the instruments is described
in this section. These instruments include a required online Annual
Partner Implementation survey and optional online materials
development and review forms. Table
3 reports the
information used to calculate the respondent burden, including the
type of instrument, number of respondents, the estimated response
rate, the estimated time each respondent will spend completing the
task. These
burden estimates
are based on sample sizes and completion rates provided in Item 1,
Table 3. Calculation of Respondent Burden
Annual GLOBE Partner Survey |
Respondents |
Hours |
Total Burden |
International Country Coordinators |
108 |
2 |
216 |
U.S. Partners |
150 |
2 |
300 |
TOTAL |
258 |
2 |
516 |
E
There are no respondent costs other than labor hours; this cost estimate is shown in Table 3.
Estimated Costs to Federal Government
2006 Partner Survey Preparation time and cost
Staff |
Tasks |
Hours worked |
Cost |
Teresa Kennedy |
Survey text development |
60 |
$5,904 |
Paula Robinson |
Survey text development |
20 |
1,343 |
Karen Milberger |
Technical support |
60 |
3,271 |
|
|
|
$10,518 |
There are no estimated costs to GLOBE Partners. They complete the survey on a voluntary basis.
Reasons for Change in Burden or Program
The total annual response hours requested has been reduced using several techniques:
By constructing online survey and assessment instruments; and
By limiting the surveys to Partners and not including teachers and their students.
P
A summary of the data gathered through the Annual Partner Survey is provided to all Partners at each Annual Conference. Individual responses to any of the items on any of the instruments are not published, except as part of summary reports and papers about the GLOBE Program and its implementation. If, as part of a description of GLOBE at a particular school, an individual is named in association with a particular response, that person's full consent will have been obtained ahead of time, and they will be given a chance to review the quotation in the context of the report in which it is presented. Absolutely no student names will be associated with individual responses at any point in the evaluation. In addition, Partners are encouraged to provide their specific results in the form of Partnership Reports published in the GLOBE Annual Conference Proceedings.
Non-display of Expiration Date
N/A.
Exceptions to the Certification Statement
N/A.
B
METHODS
1. Description of Respondent Universe and Sample and Procedures for Collection of Information
Large-scale GLOBE Partner Survey. All GLOBE Partners are required to complete the annual GLOBE evaluation form in January of each year.
2. Schedule and Procedures for Collection of Information
October-December 2005: Notification of survey
January 2006: Implementation survey covering 2005 activities
February 2007: Follow-up of non-respondents.
October-December 2006: Notification of survey
January 2007: Implementation survey covering 2006 activities
February 2008: Follow-up of non-respondents.
October-December 2007: Notification of survey
January 2008: Implementation survey covering 2007 activities
February 2009: Follow-up of non-respondents.
M
As shown above, a rigorous schedule has been developed for notifying respondents prior to the survey and for following up on non-respondents. Our experience has shown that providing notification of a survey before it is conducted and following survey dissemination with up to 3 mail, e-mail, or telephone reminders produces a very high rate of response. Because of these follow-up procedures in place, the high morale among the GLOBE partners and their teachers, and the low burden entailed in completing the online form, response rates for active GLOBE partners is 100%.
Tests of Procedures and Methods
Large-scale GLOBE partner surveys have been conducted in the past two years. Survey items have undergone minor modifications for each new use, reflecting lessons learned from prior years' data and changes in GLOBE Program content.
Persons Responsible for Data Collection Design and Development
Dr. Teresa Kennedy worked with Dr. William Penuel and his team from of SRI International to create the GLOBE Annual Partner Survey. She will lead all activities related to this survey.
Dr. Teresa Kennedy
Deputy Director, The GLOBE Program
University Corporation for Atmospheric Research (UCAR)
3300 Mitchell Lane
Boulder, Colorado 80301
303-497-2643
tkennedy@globe.gov
| File Type | application/msword |
| File Title | 12-12-06 GLOBE Program Office |
| Author | LMIT-ODIN |
| Last Modified By | Walter Kit |
| File Modified | 2006-12-19 |
| File Created | 2006-12-19 |