21st Century Community Learning Centers Study

Contract # ED-CFO-10-A-0110-0001

21^st Century Community Learning Centers: Lessons Learned Guides

Deliverable 5.3/6.3/7.3/8.3: Final Additional Materials

(Online Screening Tool & Previsit Information Survey)

January 16, 2012

Revised January 31, 2012

Revised February 6, 2012

Prepared for

Prepared by

Online Screening Tool for Site Selection

Introductory E-mail

Dear [Project Director]:

You should recently have received a letter from Stuart Kerachsky, Director of Policy and Program Studies Service at the U.S. Department of Education, informing you that your program has been identified for possible participation in a research study.

Building on its investment in the 21^st Century Community Learning Centers program, the Department has commissioned a study to learn from the field about four content-specific topic areas of afterschool and summer programming, including (1) Science, Technology, Engineering, and Math (STEM), (2) English Learners (EL), (3) Career and Technical Education (CTE), and (4) structures to increase learning time (ILT). Your program has been identified as a potential site for the study in regard to programming in [STEM/EL/CTE/ILT]. For this study, [STEM refers to teaching and learning in the science, technology, engineering, and math content areas / EL refers to students whose first language is not English and encompasses the spectrum of students from those who are just beginning to learn English, often referred to in federal legislation as "limited English proficient" (LEP), to those who have already developed considerable proficiency / CTE refers to activities that can provide students with the academic, technical, and employability skills needed to prepare for education and training beyond high school, transitions to the workplace, and long-term career options / ILT is defined as the strategy of increasing the total number of hours dedicated to engagement in core academic subjects and enrichment activities, as well as to teacher/afterschool staff collaboration, joint planning, and professional development through the alignment of out-of-school time programs with the school day.]

In order to better understand your work around […], we request that you complete an online screening tool. Please complete this online tool by [DATE]. It should take you no more than 1 hour to complete. The project team will take every measure possible to ensure the confidentiality of the data collected, to the extent possible. To reach the screening tool, please click on the following [LINK]. 

Thank you for participating in this study. As you may know, U.S. Department of Education subgrantees are required to participate in federal evaluations under ESEA, Sec. 9306(a)(4). If you have any questions or concerns, please feel free to contact me. My phone number and e-mail address are included below.

Sincerely,

Priscilla Little

Principal Investigator

[CONTACT INFORMATION]

Public Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number XXXX-XXXX. Public reporting burden for this collection of information is estimated to average 60 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is required to obtain or retain benefit (ESEA, Sec. 9306(a)(4)). Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Education, 400 Maryland Ave., SW, Washington, DC 20210-4537 or email ICDocketMgr@ed.gov and reference the OMB Control Number XXXX-XXXX. Note: Please do not return completed Screening Tools to this address.

Online Screening Tool

Instructions.

Please complete the following Screening Tool in its entirety. It should take no more than 1 hour to complete. The obligation of subgrantees to respond to the study is required, according to the requirements under ESEA, Sec. 9306(a)(4). The project team will take every measure possible to ensure the confidentiality of the data collected, to the extent possible.

Public Burden Statement: According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number XXXX-XXXX. Public reporting burden for this collection of information is estimated to average 60 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is required to obtain or retain benefit (ESEA, Sec. 9306(a)(4)). Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Education, 400 Maryland Ave., SW, Washington, DC 20210-4537 or email ICDocketMgr@ed.gov and reference the OMB Control Number XXXX-XXXX. Note: Please do not return completed Screening Tools to this address.

The first set of questions asks for some general information about[insert site name].

1. How long has the program been in operation?

2. What percentage of staff have been employed with the program for

a. Less than one year:

b. 1-2 years:

c. 3-5 years:

d. More than 5 years:

3. What is the total attendance for your program?

4. What is your percentage of regular attendees?

5. What is the student demographic for your program?

1. Percentage of students who are Caucasian:

2. Percentage of students who are African-American:

3. Percentage of students who are Hispanic:

4. Percentage of students who are Asian:

5. Percentage of students who are Native American:

6. Percentage of students who are English Learners:

7. Percentage of students who receive free or reduced price meals:

8. Percentage of students with an Individualized Education Programs (IEP):

9. Other (please specify):

The next set of questions asks about activities related to the [Topic AREA(S)] activities at [insert site].

6. How many years has your program been working to promote [Topic AREA(s)] activities?

7. Please use the following scale to describe the extent to which you use any of the following specific instructional practices for [TOPIC AREA(S)]:

• SCALE: Always / Often / Sometimes / Rarely / Never

1. For STEM:

1. Opportunities to Engage in Project-Based Activities

• Students in the program are engaged in hands-on, experiential activities

• Students in the program lead investigations to solve authentic problems

• Students in the program are engaged in projects that end with a product

• Students in the program use technology to support their work in STEM-related projects (for example, using a GPS for a mapping project or cameras/PDAs to document a science project)

• Staff in the program are intentional about the design of the activities to incorporate STEM knowledge and skills

2. Alignment With the School Day

• There is a structure in place for 21^st CCLC staff and school day teachers to communicate about students

• There is a structure in place for 21^st CCLC staff and school day teachers to share resources

• Activities/curricula in the program are aligned with national standards

• Activities/curricula in the program are aligned with state standards

• Activities/curricula in the program are aligned with school-day instruction

3. Community Partnerships

• The program uses scientists, engineers, or mathematicians (e.g., statisticians, accountants, actuaries) to provide activities

• The program uses scientists, engineers, or mathematicians (e.g., statisticians, accountants, actuaries) to expose students to STEM careers

• The program uses professional STEM educators or features professionals from STEM-oriented educational organizations to provide activities

• The program is engaged in a relationship with and uses resources from local institutions, such as museums, parks, and nature preserves

4. Professional Development in Content Area

• Professional development/training for 21^st CCLC staff on how to teach in any of the STEM areas

• Professional development on how to teach using technology to support instruction (e.g., running virtual experiments, running simulations, using GPS technology for mapping)

• Professional development on how to differentiate instruction for all levels of STEM learners

2. For EL:

1. Adequate Professional Development (PD) Specifically Related to ELs

• Staff receive PD in supporting students’ English development (e.g., by scaffolding, listening, reading, writing, speaking)

• Staff receive PD in supporting students’ general academic language development

• Staff receive PD in working with immigrant and/or refugee families

• Staff receive PD in understanding and addressing the socio-emotional needs of ELs

• Staff receive PD in supporting students in their primary language

• Staff receive PD in using EL student data to support their instruction and evaluate their progress

2. Instructional Practices Focused on Language/Academic Support

• ELs are provided structured and focused opportunities to practice using English in a variety of settings (including project-based learning) by listening, speaking, reading, and writing in English

• ELs are provided opportunities to practice using English by engaging with English speakers in a variety of contexts designed to ensure EL participation

• ELs are provided explicit language development and vocabulary development support

• ELs are provided academic support in their primary language

3. Culturally Relevant Programming

• Staff can and do speak to and support youth in their primary language

• The program provides activities and programming that build healthy cultural identities

• The program uses materials and curriculum that reflect the lives and cultures of the EL participants

• The program provides opportunities for non-ELs to formally learn the language of ELs who participate in the program (e.g., through bilingual immersion activities)

• Lessons and activities are planned to include aspects of the culture and experience of EL students

• Environmental print (e.g., posters and pictures on the wall) and other materials in the learning environment reference students’ culture or a diversity of cultures

• Programs address the socio-emotional needs specific to ELs, such as culture shock, war and other family trauma, family separation, etc.

4. Alignment With the School Day

• There is a structure in place for 21^st CCLC and school day teachers to communicate about EL students and program content/curriculum

• There are structures in place for sharing student data on level of English language proficiency and academic level (including reading, writing, speaking, and listening)

• Activities/curricula in the 21^st CCLC program are aligned with national standards

• Activities/curricula in the 21^st CCLC program are aligned with EL standards and state academic standards

• Activities/curricula in the 21^st CCLC program build on school-day instruction

5. Family and Community Engagement

• The program translates all materials in home language(s)

• Frequent EL family involvement in 21^st CCLC program

• The program conducts frequent and targeted outreach to EL families and communities in a variety of ways

• The program provides opportunities for EL families to participate in the program

• Staff speak the languages of the EL families and communities

• The program hires staff from the EL communities

• The program provides literacy services to families

6. Meaningful EL Inclusion and Affirming Learning Environments

• Staff use strategies that ensure EL comprehension and frequently test students for their comprehension

• Staff use accessible vocabulary

• Staff intentionally elicit participation from all participants, specifically ELs

• Staff intentionally create emotionally safe learning environments by discouraging bullying and teasing of ELs

3. For CTE:

1. Authentic Career-Related Learning Opportunities (e.g., apprenticeships or internships)

• The program offers hands-on, experiential opportunities to learn industry-specific and career-related skills

• The program offers project-based learning opportunities that pertain to a specific career, industry, or technology

2. Professional Development Related to CTE

• The program has targeted recruitment efforts to identify instructors meeting technical and academic requirements

• The program has specific pre-service and/or in-service training and professional development for staff in teaching career education

• The program has specific pre-service and/or in-service training and professional development in technology

• The program has planning time dedicated to discussion of and planning for CTE activities

3. Alignment With the School Day

• There is a structure in place for program staff and school day teachers, particularly CTE teachers, to communicate about students

• There are structures in place for resource and information sharing between school day teachers and program staff

• Program staff communicate and work with CTE-oriented student organizations

• There is alignment of national learning standards with CTE activities or curriculum, if applicable

• There is alignment of state learning standards with CTE activities or curriculum, if applicable

• There is alignment of school-day curriculum with afterschool CTE activities or curriculum

4. Using Data to Target Skills and Outcomes

• The program uses technical skill assessments (written exams, capstone projects, or demonstrations of skill)

• The program uses employability skill assessments

• The program has team meetings to discuss performance of individual students and prepare action plans

• The program has data-sharing agreements with institutes of higher education and employers to track student postsecondary outcomes

5. Community Partnerships

• The program uses business or other professionals to provide CTE activities at the program

• The program uses businesses or local industries to provide site-based apprenticeships and on-the-job training

• The program uses business or other professionals to expose students to careers

• The program is engaged in a formal relationship with and uses resources from local industry partners

• The program operates in a variety of settings, such as the school or a vocational center, or in the community

4. For ILT:

1. Alignment With the School Day

• Program and school have structures for communication

• Program and school have a designated person to support alignment

• Staff from program and school work in both contexts

• The program activity leaders and school day teachers share curriculum and resources

• The program staff, particularly the coordinator and/or activity leaders, and school staff engage in joint planning

• The program and school have a shared vision for success

• Program activities are aligned with national and state standards

• Program uses a blended staffing model with crossover between school and afterschool staff

• Program and school day staff participate in joint professional development experiences

2. Promoting Attendance and Removing Barriers

• The program engages in recruitment strategies that target specific students

• The program employs transportation strategies to enable sufficient participation

• The program has policies to address chronically absent participants

• The program connects to community partners to assist with recruitment

• The program seeks input from school day personnel on how to target recruitment efforts

3. Supporting Student Engagement

• Intentional efforts are made to create positive youth-adult relationships

• The program considers the developmental needs of each student

• Youth are involved in decision making

• The program communicates with parents about the importance of participation

• The program supports families’ efforts to promote student achievement

4. Instructional Practices Promote Increased Time for Learning

• Project-based learning is used as an instructional approach

• Collaborative peer learning is used as an instructional approach

• Learning settings are designed to best meet student needs (e.g., group, one-on-one, peer)

• The program coordinates instruction with school day teachers

• The program is able to articulate similarities and differences with the school day

5. Using Information to Evaluate Program Implementation and Student Outcomes

• The program and school have a formal agreement to share student test scores and other relevant information

• The program has data coaches to help staff use school day information

• The program and school use student data, such as test scores, to target specific skills in need of remediation or support

• There are structures in place for the program and school to communicate about student progress

• The program has an “early warning system” of examining student attendance, grades, and behavior

• The program engages in assessment for continued program improvement and/or external evaluation

6. Leveraging Partnerships

• The program engages community partners to increase learning time

• The program partners are actively involved in the implementation of other key practices

8. Please provide one or two examples of the practices that you feel best illustrate your approach to [TOPIC AREA(S)]. (open-ended)

9. On average, how frequently do you offer activities related to [TOPIC AREA(S)] in any given week?

• 1 day per week

• 2 days per week

• 3 days per week

• 4 days per week

• 5 days per week

• [Topic area] is embedded in most activities

10. Where do the activities related to [TOPIC AREA(S)] occur?

• Only at the program site

• At the program site and in the community

• Only off-site

• Online (remote/virtual)

We are also interested in learning more about the students who participate in your [Topic AREA(S)] activities.

11. What is the target age group for the activities related to [TOPIC AREA(S)]? (Please check all that apply.)

• Elementary age

• Middle school age

• High school age

12. Do you target specific populations for the program? If so, which ones and how? (open-ended)

13. If you target high school students, what kind of leadership opportunities do they have while participating in [TOPIC AREA(S)] activities?

• Leadership opportunities to design and/or lead activities

• Paid internships

• Other (please describe):

14. On average, how many students participate in the [TOPIC AREA(S)] activities at your program?

	1-10 students	11-20 students	21-40 students	41-60 students	Over 60 students
Elementary School
Middle School
High School

15. What is the staff/student ratio for your program’s [TOPICAREA(S)] activities, by age group?

	Elementary School	Middle School	High School
Staff/student ratio

Now we are interested in learning about the activity leaders for your [Topic AREA(S)] activities.

16. Who are the activity leaders for the [TOPIC AREA(S)]? (Please check all that apply.)

• Full-time program staff person

• Part-time program staff person

• School day staff person

• Paid expert from the community

• High school or college age students

• Volunteer

• Other (please describe):

17. What kind of orientation do you have for new [Topic AREA(S)] activity leaders? (Please check all that apply.)

• Formal (written materials and formal meetings)

• Informal (verbal orientation with director or other staff)

• Specific training on [TOPIC AREA(S)] (please describe)

• No orientation for [TOPIC AREA(S)] activity leaders

18. Please describe how the [Topic AREA(S)] activity leaders are supervised, if at all. (Please check all that apply.)

• Regular site supervisor observations with feedback

• Formal annual performance review

• No supervision of [Topic AREA(S)] activity leaders

19. How often do the activity leaders for [Topic AREA(S)] meet to discuss program-related issues?

• Never

• Once a year

• Every two to three months or once a semester

• Monthly

• Twice a month

• Weekly

Last, we would like to know more about the outcomes you hope to achieve by implementing [Topic AREA(S)] activities.

20. What kinds of outcomes are you hoping to achieve by implementing activities related to [TOPIC AREA(S)] and what progress have you made in achieving these outcomes? (Please select all that apply.)

• Improved grades

• Improved scores on standardized tests

• Improved attendance

• Improved on-time grade promotion/graduation rates

• Improved confidence and competence in [TOPIC AREA(S)]¹

• Improved behavior

• Improved attitudes toward school and learning

• Other (please specify):

21. What progress have you made in achieving these outcomes and what sources of evidence do you have for this progress? (Please select all that apply.)

Outcomes	Progress to Date			Sources of Evidence
	Limited Progress	Moderate Progress	Strong Progress	Informal Evidence of Improvement (Please see below for examples)	Formal Evidence of Improvement (Please see below for examples) Please specify source(s)
Improved grades
Improved scores on standardized tests
Improved attendance
Improved on-time grade promotion / graduation rates
Improved confidence and competence in [STEM/EL]
Improved behavior
Improved attitudes toward school and learning
Other (please specify):

Examples of informal evidence:

• Gather informal youth feedback

• Gather informal parent feedback

• Informal assessments

Examples of formal evidence:

• Track program attendance

• Gather formal youth feedback (surveys, etc.)

• Gather formal parent feedback (surveys, etc.)

• Gather formal program staff feedback

• Gather formal school staff feedback

• Formal assessments

  • Formative assessments (ongoing)

  • Summative assessments (end of year)

• External evaluation

Thank you for completing this survey!

Previsit Information Survey

Notification E-mail

Dear [Project Director]:

As you know, your program has been selected for participation in a research study of 21^st CCLCs. Your program is one of only 15 programs that we will be visiting across the country.

Building on its investment in the 21^st Century Community Learning Centers (21^st CCLC) program, the Department has commissioned a study to learn from the field about four content-specific topic areas of afterschool and summer programming, including (1) Science, Technology, Engineering, and Math (STEM), (2) English Learners (EL), (3) Career and Technical Education (CTE), and (4) structures to increase learning time (ILT). Your program has been selected as an example of a 21^st CCLC program that demonstrates strong promise in [STEM/EL/CTE/ILT].

This study will highlight the work of selected 21^st CCLC programs in four “Lessons Learned Guides” that may help 21^st CCLC programs nationwide improve and refine their practices.

A two-day site visit will be scheduled for some time between April 2012 and November 2012. In addition to the information you previously provided in the online screening tool, we would like to ask you to complete an online information survey. This survey has been pre-populated wherever possible with information that you provided in the screening tool during the site selection process. Please verify the accuracy of this information and complete any questions that have been left blank. It should take you no more than 30 minutes to complete. The project team will take every measure possible to ensure the confidentiality of the data collected, to the extent possible. The obligation of grantees to respond to the study is required, according to the requirements under ESEA, Sec. 9306(a)(4). The survey can be found at [LINK]. We ask that you please complete it by [DATE].

Congratulations on your selection, and I thank you and your staff for supporting this study. We will be in touch with you by phone to begin the process of scheduling the dates for the visit and arranging other logistics. If you have any questions or concerns, I can be contacted at the phone number or e-mail address included below.

Sincerely,

Priscilla Little

Principal Investigator

[CONTACT INFORMATION]

Public Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number XXXX-XXXX. Public reporting burden for this collection of information is estimated to average 60 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is required to obtain or retain benefit (ESEA, Sec. 9306(a)(4)). Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Education, 400 Maryland Ave., SW, Washington, DC 20210-4537 or email ICDocketMgr@ed.gov and reference the OMB Control Number XXXX-XXXX. Note: Please do not return completed Information Surveys to this address.

Information Survey

In preparation for our team’s visit to your site, we would like you to please provide us with the information requested below. This survey has been pre-populated wherever possible with information that you provided in the screening tool during the site selection process. Please verify the accuracy of this information and complete any questions that have been left blank. It should take no more than 30 minutes to complete. The obligation of subgrantees to respond to the study is required, according to the requirements under ESEA, Sec. 9306(a)(4). The project team will take every measure possible to ensure the confidentiality of the data collected, to the extent possible.

This survey will allow us to better understand the basic structure and conditions of your program so that we can best target our interviews and observations during the visit. We ask that the form be filled out by the program director, site coordinator, or another person familiar with the workings of the program overall. Thank you for your time in completing this preliminary information survey.

Public Burden Statement: According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number XXXX-XXXX. Public reporting burden for this collection of information is estimated to average 60 minutes per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is required to obtain or retain benefit (ESEA, Sec. 9306(a)(4)). Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Education, 400 Maryland Ave., SW, Washington, DC 20210-4537 or email ICDocketMgr@ed.gov and reference the OMB Control Number XXXX-XXXX. Note: Please do not return completed Information Surveys to this address.

21^st CCLC Practitioner Guides Pre-visit Information Survey²

1. Program name:

2. Contact:

3. Title:

4. Physical address:

5. Phone:

6. E-mail:

Overall Program Data

7. Program enrollment:

8. Percentage regular attendees:

9. Target attendance:

10. Age breakout of participants (elementary/middle/high school):

11. Student demographic breakout:______________

1. Percentage of students who are Caucasian:

2. Percentage of students who are African-American:

3. Percentage of students who are Hispanic:

4. Percentage of students who are Asian:

5. Percentage of students who are Native American:

6. Percentage of students who are English learners:

7. Percentage of students who receive free or reduced price meals:

8. Percentage of students with an Individualized Education Programs (IEP):

9. Other (please specify):

12. Hours and days of operation: __________

13. Duration of program (i.e. semester, school year, summer, etc.):________________

14. How long the program has been in operation:_____________

15. Average number of days that students participate per week: _____

16. Average number of hours students participate per week: ___

17. Major program partnerships:______________________

18. Nature of partnerships:___________________________

19. Duration of partnerships:__________________________

[TOPIC AREA(S)] Data*

20. Number of students enrolled/participating in [TOPIC AREA(S)]:

21. Age breakout of [TOPIC AREA(S)] participants (elementary/middle/high):

	1-10 students	11-20 students	21-40 students	41-60 students	Over 60 students
Elementary School
Middle School
High School

22. Percentage regular attendees for [TOPIC AREA(S)] activities:

23. Target attendance for [TOPIC AREA(S)] activities:

24. Hours and days when [TOPIC AREA(S)] activities are offered: ________________

25. Average number of days students participate per week in [TOPIC AREA(S)] activities: _____

26. Average number of hours students participate per week in [TOPIC AREA(S)] activities:

27. Major [TOPIC AREA(S)] partnerships:______________________

28. Nature of [TOPIC AREA(S)] partnerships:___________________________

29. Duration of [TOPIC AREA(S)] partnerships:__________________________

Budget

30. Total program budget:

31. Non-21^st CCLC funding sources:

32. Major nonmonetary funding (in-kind donations, volunteer labor hours, etc.):_____________

33. Program budget dedicated to [TOPIC AREA(S)], if available*:____________

34. Non-21^st CCLC funding sources for [TOPIC AREA(S)], if available*:___________

35. Major nonmonetary funding (in-kind donations, volunteer labor hours, etc.) for [TOPIC AREA(S)], if available*:______

Staffing

36. Is the director full-time or part-time? ______

37. Is the site coordinator full-time or part-time?_____

38. Please complete the following table:

Staffing Questions		General Staffing	[TOPIC AREA(S)] Staffing*
Number of full-time staff
Number of part-time staff
Approximate number of weekly hours worked by part-time staff (per staff members)
Number of volunteers
Number of high school and college age students
How many staff have been employed with the program for
	Less than one year
	1-2 years
	3-5 years
	More than 5 years

Documentation

Please submit any program documents that would help us better understand your organization. Relevant program documents include, but are not limited to:

• Curricula/lesson plans/teacher guides in [TOPIC AREA(S)]

• Handouts and/or forms relating to [TOPIC AREA(S)]

• Brochures/flyers about the program or specific [TOPIC AREA(S)] activities within the program

• Attendance data (without student names or other identifying information)

• Websites developed by staff/youth that relate to [TOPIC AREA(S)]

• Formal external evaluations

• Results of formal surveys or focus groups

• Self-assessment reports

• Press clippings

• Testimonials from partners

Program documents that are available in electronic format can be e-mailed directly to [E-MAIL ADDRESS]. Program documents that are available only in hard copy will be collected by the site visit team during the site visit. Please use only pre-existing documents; do not create new documentation for the purposes of this survey.

39. Please list those documents you will be submitting by e-mail:_____________

40. Please list those documents you will be submitting in hard copy during the visit:_________

1 Please note: Only to be used for STEM and EL

2 Please note: Those questions or sections marked with an asterisk (*) will be asked only for STEM, CTE, and certain EL sites—those with a dedicated EL program (versus sites with EL populations but no dedicated EL program).