1205-0436 SS Part B final 2013

1205-0436 Supporting Statement Part B

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1.       Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used.  Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample.  Indicate expected response rates for the collection as a whole.  If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Depending on the survey, the administration will be either to the universe of respondents or to a stratified random sample.    Drawing samples using stratified random selection in such circumstances can improve the efficiency of sample estimates when the dimension on which the sample is stratified is strongly related to the research question being examined.  Given that administrative and governance structures and service design and delivery will encompass the most likely research questions being investigated under this clearance, we propose that program size is the most reasonable stratification criterion.  

On this basis, Table 3 offers an example of sampling rates and strategies for drawing samples of states, local areas, and One-Stop Centers. The calculations in Table 3 below are based on an upper limit of 250 respondents; however, it is understood that an occasional survey may exceed that upper limit by no more than 370 respondents. That total number of 620 is derived from adding together the total number of State and local workforce investment boards.      A calculation of the precision and necessary sample size will be done for each survey and provided to OMB for approval.  Details of minimal substantively significant effect sizes will be included in the OMB submission for each survey.

Table 3

 

Universe		# of Respondents		How Drawn
State agencies		All in Universe		All in universe

 

Local workforce areas (LWIAs)		250 LWIAs (of 566 in universe)		Stratified random selection.  Five strata will be defined.  After ranking LWIAs by amount of funding, each stratum will be defined such that each will make up equal proportions of total WIA funding.  Thus, the first stratum will include the largest LWIAs that make up the top 20% of aggregate WIA funding nationwide; the next stratum will include LWIAs that make up the next 20% of funding, and so on.  An equal number of LWIAs (50, or 250/5) will be selected randomly from within each stratum.
American Job Centers		250 Centers (of approximately 3,300 in universe)		Stratified random selection of 250 LWIAs will be selected using the method described above.  One Center will be randomly selected within each of these LWIAs using a simple random sampling.

 

We expect to receive a response rate of at least 80% for all surveys associated with this information collection plan.  This estimate is consistent with results from previous data collection efforts of similar nature and magnitude, including the Job Corps High School Diploma Initiative Survey administered by the National Office of Job Corps, which received an 82% response rate.

 

2.         Describe the procedures for the collection of information including:

 

        Statistical methodology for stratification and sample selection,

        Estimation procedure,

        Degree of accuracy needed for the purpose described in the justification,

        Unusual problems requiring specialized sampling procedures, and

        Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

For each survey to be conducted, the OMB package will give a very detailed point-by-point statement of the population(s) of interest, the population parameters (including formulas) about which the ETA wishes to make inferences, the full sample design, the data collection instrument to be used, point estimation and variance estimation methods to be used (again, including formulas), and nonresponse adjustment or imputation methods, if any, to be used in the analysis.

For a typical survey:

a) Sample Selection.  Sample selection will be carried out using the procedures described above, in answer to question 1.

b) Estimation Procedures.  The analysis will primarily make use of frequency distributions and cross-tabulations, which will provide basic information about agency opinions and service strategies.  Basic characteristics of all LWIAs are known from existing sources (e.g., their level of funding, types of persons served, type of administrative structure, location, etc.); the data can be used to suggest whether non-respondents differ in any substantial way from respondents and to develop weights for respondents in order to accurately depict the frequencies or cross-tabulations.

c) Degree of Accuracy.  For surveys directed to state agencies, sampling will not be employed.  Thus, results should be an accurate reflection of the relevant universe, subject to the constraints of reporting error, after appropriate adjustments for non-response bias are made.  For surveys directed to LWIAs and American Job Centers, we may select a sample of 250 respondent entities, as described above.  However, on occasion, we may choose to sample all of the LWIAs and American Job Centers. Assuming a response rate of 80% would yield completed interviews for 200 entities. 

The degree of accuracy for a survey directed to LWIAs or One-Stop centers will be determined as follows. Suppose H strata of sizes N_h, h = 1, 2, …, H, from each of which n_h units are sampled respectively by simple random sampling (srs).  Suppose the proportion of “successes” in the hth stratum is p_h.  Then the overall proportion of successes in the population is

,

where N is the total population size and S is the total number of successes in the population.

 

An unbiased estimator of p is

,

where , s_h  being the number of successes in the sample from the hth stratum, so that  is an estimate of the number of successes in the population.

 

The variance of this estimator is given by

 

.

d) Unusual problems.  There are no unusual problems.

e) Periodic Data Collection.  Each survey will be administered only once.

3.   Describe methods to maximize response rates and to deal with issues of non-response.  The accuracy and reliability of information collected must be shown to be adequate for intended uses.  For collections based on sampling, a special justification must be provided for any collection that will not yield reliable data that can be generalized to the universe studied.

In order to maximize response rates and minimize non-responses, ETA’s contractor will attempt to contact respondents multiple times until a completed survey is obtained.  When web surveys are used primarily, the follow-up will be by telephone; an attempt to administer the survey by telephone will be attempted on the spot. 

When phone surveys are used primarily, the contractor will attempt to reach each respondent by phone multiple times.  Should this approach not yield a completed survey, telephone calls will be made to another contact person within the targeted agency.  

Basic characteristics of state agencies and of LWIAs are known from existing sources (e.g., their level of funding, types of persons served, type of administrative structure, location, etc.).  These data can be used to suggest whether non-respondents differ in any substantial way from respondents and to develop weights for respondents in order to accurately depict the frequencies or cross-tabulations.  Because the sample will be drawn using probabilistic selection methods, results will be generalized to the universe studied after making appropriate corrections for potential selection bias.

4. Describe any tests of procedures or methods to be undertaken.  Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility.  Tests must be approved if they call for answers to identical questions from 10 or more respondents.  A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

ETA’s contractor will pretest each survey with up to, but no more than, nine respondents.  The pretest will assess the clarity of content and wording of the survey, the organization and format of the questionnaire, respondent burden time, and potential sources of response error.  The pretest will be used to modify the questionnaire as appropriate.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

ETA’s contractor(s) will refine the design and carry out the sampling plan, collect the data, and prepare data tabulations and brief analyses.  The contractor that ETA uses will vary from survey to survey.  However, the contractors will be selected because their substantive background makes them very familiar with the context of the questions being asked and because of their expertise in survey design and administration and statistical methods. Information about any individuals consulted on statistical aspects of the survey design will be included in the OMB submission for each survey. Contact information for the collecting agency will also be included.

 

3