1875-NEW prev 4559 OMB Supporting Statement B revised 10-21-11

E

POLICY STUDIES ASSOCIATES, INC.

POLICY STUDIES ASSOCIATES, INC.

VALUATION OF THE EDUCATION

FOR HOMELESS CHILDREN AND
YOUTH PROGRAM–SUPPORTING

STATEMENT FOR PAPERWORK REDUCTION

Part B: Collection of Information Employing

Statistical Methods

Prepared by:

Policy Studies Associates

1718 Connecticut Ave., NW

Suite 400

Washington, DC 20009

Prepared for:

U.S. Department of Education

Policy and Program Studies Service

Office of Planning, Evaluation, and Policy Development

O ctober 14, 2011

Supporting Statement for Paperwork Reduction Act Submissions

B. Collection of Information Employing Statistical Methods

B.1. Respondent Universe and Sample Size

The U.S. Department of Education (ED) contracted with Policy Studies Associates (PSA) to administer two surveys: one to State Coordinators in each of the 50 states, the District of Columbia, and Puerto Rico and one to a sample of liaisons in local education agencies (districts) that receive Education for Homeless Children and Youth (EHCY) Program subgrants.

The population of districts that received regular EHCY Program subgrants in FY 2011 will provide the population from which the evaluators will draw the sample. The list of FY 2010 ARRA EHCY Program awardees will help to validate state survey responses on questions related to ARRA funding, and it will serve as an independent variable in statistical analyses to measure the effect on implementation of the additional ARRA funding.

In the 2010- 2011 school year, excluding those districts that do not directly educate students (e.g., regional education service agencies), that serve educated special populations of students (e.g., vocational districts), or that are charter agencies, there were 3,079 unique districts that reported receiving EHCY Program subgrants. PSA will purposefully sample the 50 very large districts in the treatment group sample frame in order to ensure enough precision to draw conclusions about these largest districts and then draw a simple random sample of 451 districts from the remaining districts for a total sample of 501 McKinney-Vento funded districts. Achieving a response rate of 85 percent requires responses from approximately 426 participating districts. Although evaluators hope to achieve a response rate of 100 percent, an 85-percent response rate will allow an estimate of population values with confidence intervals of approximately 5 percent and allow inferences about the population values with a confidence level of 95 percent.

Exhibit 1: Universe and Sample Sizes of Respondent Groups

	Universe	Sample	Expected response rate, in percents	Response rate of 2002 data collection, in percents
State Survey	52	52	95	95
District Survey Subgrantees	3,079	551	85	N/A

PSA will use the National Center of Education Statistics’ Common Core of Data (CCD) Local Education Agency Universe Survey to stratify the population by geographic location and area population density (using the ULOCAL code) prior to drawing the sample.

B.2. Procedures for Collection of Information

ED and the National Center on Homeless Education (NCHE) will assist in creating a list of state coordinators and their relevant contact information to take the state survey and, once a sample of district subgrantees and matched non-grantees have been selected, will assist with identifying the district liaisons and their contact information in subgrantee districts.

Survey administration will begin in January 2012 after OMB clearance has been confirmed. Prior to conducting the surveys, state coordinator and district liaison respondents will be notified about the study via an introductory letter. PSA will also mail a study notification letter to Chief State School Officers and district superintendents to make them aware of the evaluation study and to enlist their support in ensuring a high response rate. The introductory and notification letters, signed by an ED official, will explain the evaluation and its importance to ED. They will also indicate that appropriate OMB clearance has been obtained to administer the survey and that ED’s EHCY program office supports this survey and that participation is mandatory.

Each recipient will receive a follow-up e-mail that again explains the study and contains an individualized link to the web-based survey. To secure the cooperation of respondents and to achieve a high response rate (85 percent or more for districts and 95 percent or more for states), respondents will be able to complete the state in approximately 35 minutes and the district survey in approximately 30 minutes.

A week before the survey is due, PSA will mail a reminder postcard to non-respondents, with instructions for requesting another individualized link or a paper survey. A week after the due date, PSA will again e-mail individualized links to all non-respondents, with instructions for requesting paper surveys. Two weeks later, PSA will mail paper surveys to non-respondents and will begin telephone follow-up. Each non-respondent will receive up to three telephone calls, asking him or her to complete the survey and offering to send replacement links or paper surveys. PSA staff will offer to walk respondents through the survey and collect responses by telephone. For those districts that still have not responded, PSA will contact the relevant state coordinators to enlist their assistance in urging district administrators to participate.

In February 2011 ED discussed the survey and evaluation at the state coordinators meeting to explain what is expected of state coordinators. State level grantees and district level subgrantees are required to participate in ED evaluations under ESEA, Sec. 9304(a)(4) and ESEA, Sec. 9306(a)(4), respectively.

B.3. Methods for Maximizing Response Rate and Dealing with Nonresponse

The data collection procedures described in Section B.2. include a follow-up plan to ensure a high response rate.

Each recipient will receive a follow-up e-mail that again explains the study and contains an individualized link to the web-based survey. To secure the cooperation of respondents and to achieve a high response rate (85 percent or more for districts and 95 percent or more for states), respondents will be able to complete the state in approximately 35 minutes and the district survey in approximately 30 minutes.

A week before the survey is due, PSA will mail a reminder postcard to non-respondents, with instructions for requesting another individualized link or a paper survey. A week after the due date, PSA will again e-mail individualized links to all non-respondents, with instructions for requesting paper surveys. Two weeks later, PSA will mail paper surveys to non-respondents and begin telephone follow-up. Each non-respondent will receive up to three telephone calls, asking him or her to complete the survey and offering to send replacement links or paper surveys. PSA staff will offer to walk respondents through the survey and collect responses by telephone. For those districts that still have not responded, PSA will contact the relevant state coordinators to enlist their assistance in urging districts administrators to participate.

In February 2011 ED discussed the survey and evaluation at the state coordinators meeting to explain what is expected of state coordinators. State level grantees and district level subgrantees are required to participate in ED evaluations under ESEA, Sec. 9304(1)(4) and ESEA, Sec. 9306(a)(4), respectively.

B.4. Test of Procedures and Methods

ED and NCHE identified six state coordinators to provide initial feedback on the state and district surveys for the evaluators. The evaluators spoke with these state coordinators at the state coordinators’ meeting in February 2011. During March and April 2011 evaluators piloted the districts survey with less than nine district liaisons identified by ED and NCHE, while the survey was in the first stage of OMB review.

Participants in the survey pilot test were asked to track the time required to complete the survey and respond briefly to several questions related to item clarity and structure. The evaluators debriefed the participants by telephone to learn more about the respondent’s comprehension of each item’s text, definitions of key terms, precision and completeness of existing response options, and whether important questions are missing. The evaluation team analyzed participant responses item by item in a database and made survey revisions. They were incorporated into the final draft of the survey.

B.5. Consultations on Statistical Aspects of the Design

Name	Organization	Title	Phone
Stephen Coleman	Policy Studies Associates	Senior Research Associate	202-939-5306
Mark Low	Policy Studies Associates	Research Associate	202-939-5316