Supporting Statement For 1205-0436

SUPPORTING STATEMENT FOR

PAPERWORK REDUCTION ACT 1995 SUBMISSION

INFORMATION COLLECTION PLAN FOR

QUICK-TURNAROUND SURVEYS OF WIA

A.        JUSTIFICATION

1.  Explain the circumstances that make the collection of information necessary.  Identify any legal or administrative requirements that necessitate the collection.  Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

In December 2002, OMB granted a one-year approval to ETA for conducting quick turnaround surveys of activities and operations under the Workforce Investment Act (WIA) (OMB Control No. 1205-0436, expiring 12/31/03).  The authority  has been extended and is currently scheduled to expire on April 30, 2007   This new request is for an extension of the clearance to conduct eight (8) to twenty (20) surveys over an additional three years.   Section 172 of WIA is the authority by which ETA will collect the information proposed in this extension of the Quick Turnaround Surveys of WIA.

Under the existing clearance, ETA has developed a survey on services and outreach to business and industry within local One-Stop systems.  A questionnaire has been cleared by OMB.  Other surveys are under consideration, including ones possibly on youth services and another on H-1B training grants.

ETA, in its role of providing broad program oversight and policy development, needs accurate, timely information on how services and systems under WIA are unfolding and on the challenges and successes states and local areas encounter.  Only in this way can it properly discharge its obligations to issue policy clarifications, regulations and technical assistance.   This need is particularly acute given that the workforce development system has been evolving rapidly in the several years since WIA was enacted.  It is expected that WIA will continue to change rapidly, as Congress is currently considering its reauthorization, with multiple potential changes.

However, much of the information available to ETA on key operational issues is impressionistic or anecdotal in nature, based on hearsay or unsystematic observations, and not accurate as to the incidence or scope nationally.   When accurate nationwide information is available, as from long-term in-depth evaluation studies, it is often not timely.  Thus ETA has a need for accurate and timely information that can be found only with systematic quick turnaround studies.

According to the plan, eight (8) to twenty (20) surveys would be administered over the next three years, with each survey being simple and relatively short (10-30 questions).  Depending on the nature of the questions, the surveys would be administered at the state level or at the local workforce area level or some combination of both together, and might entail surveys of state workforce agencies, local Workforce Boards, One-Stop Centers, Employment Service offices, and offices of other local-area partners.  Each survey will be designed on an ad hoc basis over the three-year period, and will focus on emerging topics of pressing policy interest.  Questions would build on areas raised by ETA National Office and Regional Staff, OMB itself or Congress.   .

Examples of broad topic areas include:

        Local Management Information System developments

        New processes and procedures

        Services to different target groups

        Integration and coordination with other programs

        Local Workforce Investment Board membership and training

In accordance with the existing terms of clearance, DOL would follow the clearance process below for each individual survey:

(1) ETA will submit the following materials to OMB for review:

        OMB 83-C Paperwork Reduction Act Change Worksheet

         Description & purpose

        Burden estimates & federal costs

        Statistical methodology

        BLS reviews

        Survey instrument

        A current inventory of surveys approved under the OMB control number assigned to this data collection, displaying the number of respondents, burden hours and response rates for those approved.

 

(2) OMB will review the materials and submit any questions or comments to ETA within 15 working days of the submission to OMB. 

(3) OMB and ETA will then work collaboratively to resolve any outstanding issues. 

2.       Indicate how, by whom, and for what purpose the information is to be used.  Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

Quick turnaround surveys fill a critical gap in ETA’s information needs about how the workforce system is unfolding and inform development of legislation, regulations and technical assistance. 

  

Information from the planned survey of services and outreach to business and industry (now in process) will be used by a number of offices in ETA, such as the Business Relations Group, both for planning technical assistance and for identifying baseline levels. To date, two surveys have been submitted for review and received approval by OMB.  The availability of this quick turnaround mechanism has generated significant activity within ETA as knowledge of its availability as a resource and its possible uses has expanded. Some survey efforts could not avail themselves of the mechanism because of failing to meet one of the conditions.  Specifically, a proposed survey of the extent to which Workforce Investment Boards subcontracted with community and faith-based organizations exceeded the 250 respondent limit. Although other surveys are still boarding the planning stage, ETA anticipates that, as knowledge about this survey tool spreads more widely, demand for its use will increase significantly.                                 

3.       Describe whether, and to what extent, the collection of information involves  use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection.  Also describe any consideration of using information technology to reduce burden.

Consistent with the quick-turnaround basis underlying this data collection, surveys will be administered either via electronic submission on the Internet or by telephone.  The Internet will be used as a way of reducing respondent burden and increasing the efficiency of data collection and processing.   It is anticipated that at least 15 percent of the respondents will reply electronically.

4.       Describe efforts to identify duplication.  Show specifically why any similar information already available cannot be used or modified for use for the purposes described in item 2 above.

The information being requested is not otherwise available.  Other research and evaluation efforts, including case studies or long-range studies, either cover only a limited number of sites or take many years for data to be gathered and analyzed.   Administrative information and data are too limited:  The five-year Workforce Investment Plans, submitted by states and local areas, are too general in nature to meet ETA’s informational needs and need be updated only once every five years; mandated quarterly or annual reporting requirements of states and local areas provide some information, but primarily about cost outlays and the number and characteristics of clients served and their outcomes; and participant outcome data does not provide information on key operational practices and issues.  Thus, ETA has no alternative mechanism for collecting information that both identifies the scope or magnitude of emerging WIA implementation issues and provides the information on a quick-turnaround basis.

ETA will make every effort to coordinate the quick-turnaround surveys with other studies.  ETA will make sure there will be properly drawn samples of local sites but will avoid, where possible local areas that have already been previously contacted in other studies.   ETA will also make efforts to ease the burden on the universe of state level respondents and will make sure that interim data and information from each study will be shared and used to inform the other studies. The proposed quick turnaround surveys will provide “just-in-time” information that responds to specific issues that arise and will supplement and complement but not duplicate other ETA evaluation efforts and program reporting requirements.

5.       If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden.

The collection of information has no impact on small businesses or other small entities. 

6.       Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles in reducing burden.

 

Without the information it will collect through the proposed surveys, ETA will be less effective in carrying out its oversight and technical assistance functions and informing the legislative development process.  These functions depend on ETA’s staying abreast of emerging challenges or impediments to the delivery of workforce services under WIA.  Only with such information can it effectively issue policy guidance or clarifications, work with Congress in suggesting legislative amendments to the Act, or identify and redress technical assistance needs that states or local areas may have. 

7.       Explain any special circumstances that would cause an information collection to be conducted in a manner:

        requiring respondents to report information to the agency more often than quarterly;

        requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

        requiring respondents to submit more than an original and two copies of any document;

        requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

        in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

        requiring the use of statistical data classification that has not been reviewed and approved by OMB;

        that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

        requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

There are no special circumstances that would cause this information collection to be conducted in any manner listed above.  DOL may issue surveys more frequently than quarterly, but each survey will focus on separate issues and be administered to different respondents.   This collection of information complies with the requirements of 5 CFR 1320.5.

8.       If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB.  Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.  Specifically address comments received on cost and hour burden.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years -- even if the collection of information activity is the same as in prior periods.  There may be circumstances that may preclude consultation in a specific situation.  These circumstances should be explained.

The public was given sixty days’ comment period (Federal Register Notice

Volume 71, No. 230, dated November 30, 2006) on this information collection, with comments due by January 29, 2007. One comment was received.  The comment as summarized and the agency’s responses to the comment are below:

Comment: The current authorization does not reflect today’s environment of accountability. The structure is open-ended and does not demonstrate need. The burden is substantially underestimated, and methods for assuring quality, utility and clarity of the information to be collected are not addressed. 

Response:  ETA’s need for quick turnaround information is absolutely critical because WIA has devolved decision-making authority on many areas of policy and primary responsibility for designing and implementing programs to the State and local levels. If ETA is to carry out its obligations to issue policy clarifications, regulations and technical assistance, functions that are central to its role of providing broad program oversight and policy development, it is vital that ETA have accurate, timely information on how services and systems under WIA are unfolding.

Every survey instrument that involves statistical methodology must be reviewed beforehand by the Bureau of Labor Statistics (BLS); BLS subjects these instruments to painstaking scrutiny before allowing them to proceed further with the clearance process. This review comprehensively addresses all issues relating to statistical methodology surrounding the surveys to assure that the questions posed meet the needs of the survey.  Also the package that is sent forward to OMB provides a detailed point-by-point statement of the full sample design and data collection instrument to be used. 

Comment: Modify the Information Collection Request as follows:

• Survey respondents would be limited to State workforce agencies only;

• ETA would be allowed to administer 2 quick turnaround surveys per program year, with no more than one survey to be administered in a given quarter;

• Administration of the surveys should be scheduled to not interfere with quarterly and/or annual reporting and data element validation, to help ease the burden of responding;

• Surveys will be administered via the web;          

• State workforce agencies would be allowed opportunity to comment on the survey prior to it being administered to address concerns regarding quality, utility, clarity, and burden of the information to be collected.

Response:  As noted above, WIA devolved major decision making authority in many policy and program areas to State and local areas. Therefore ETA must be able to gain information from local areas directly as well as from States. As the supporting statement notes, the information being requested is not otherwise available.  Other research and evaluation efforts, including case studies or long range studies, either cover only a limited number of sites or take many years for data to be gathered and analyzed.  There is no alternative vehicle for collecting information that both identifies the scope and magnitude of emerging WIA issues and provides the information on a quick turn around basis.

ETA is extremely sensitive to the ongoing responsibilities of State and local entities in the WIA system, and pledges to make every effort to keep any additional burden resulting from these quick turn around surveys to a minimum. For example, ETA will make sure that there will be properly drawn samples of local sites, but will avoid, where possible, local areas that have already been contacted in other studies. ETA will also make every effort to ease the burden on the universe of State-level respondents.

Furthermore, ETA has voluntarily placed a limit of 8 to 20 surveys over a 3 year period on the quick turnaround mechanism. This upper limit is in deference to State and local responsibilities. Placing a limit of 2 surveys a year (with no more than one per quarter) would be too stringent a limit on what is ETA’s only way of gaining accurate and timely information on an ad hoc basis.

ETA agrees that the Internet is a highly effective way of conducting surveys and reducing respondent burden and is very much in the spirit of quick turnaround surveys. This is why we have anticipated that at least 15 percent of respondents will respond in this fashion. ETA would like to retain the flexibility of also using the telephone. With a stated burden of 1-3 minutes per question and a limit of 10-30 questions per survey, a properly conducted one time telephone interview would not place an undue time constraint on a respondent.

The premise that quick turn around surveys are based on is just that – short, rapid clearance and administration. Even within the quick turn around structure, there are two significant reviews – one at BLS where there are issues of statistical methodology and the other at OMB. Those two reviews should be ample to ensure the goals of quality, clarity, utility, with minimal burden. Creating another level of review by State agencies would stretch out the review process so as to eliminate the desirable quick turn around feature to these surveys.   

9.       Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

This item is not applicable.  No payment or gift to respondents will occur.

10.   Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

Persons to whom the questionnaire is distributed will be assured that their cooperation is entirely voluntary and that their responses will be held in confidence.  Data collection will be carried out by ETA’s contractors and will be released to the public at large in aggregated form only, so that no specific state or local area agency will be identified.  ETA’s contractors will be instructed to follow rigorous procedures for assuring and maintaining respondents’ confidentiality.  In keeping with this, access to any data with identifying information will be limited only to ETA staff and contractor staff directly working on the survey.

In keeping with the Confidential Information Protection and Statistical Efficiency Act of 2002, when applicable the following statement will be displayed on each questionnaire:

“The Employment and Training Administration, its employees and agents, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law.  In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (title 5 of Public Law 107-347) and other applicable Federal laws, your response will not be disclosed in identifiable form without your informed consent.”

11.     Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.  This justification should include the reasons why the agency considers these questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

None of the questionnaire items will involve sensitive content, such as the topics referenced in this question.

12.     Provide estimates of the hour burden of the collection of information.  The statement should:

Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated.  Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates.  Consultation with a sample (fewer than 10) of potential respondents is desirable.  If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance.  Generally, estimates should not include burden hours for customary and usual business practices.

This clearance package represents an umbrella clearance for 8-20 separate surveys to be conducted over a three-year period to a range of potential respondents, including (depending on the survey) state workforce agencies, local workforce investment areas, One-Stop operators, ES offices, and others.  Although each survey conducted under this clearance will be short, the exact number of questions will vary, within the range of approximately 10-30 questions per survey.   Because of this variability in both the targeted number of respondents and questionnaire length, burden estimates for each survey will vary widely.  As a way of bounding these estimates, Table 1 presents two scenarios.  In the first scenario, representing a lower-bound estimate of burden, the universe represents state workforce agencies.  For surveys to be administered to state agencies, sampling will not be conducted; instead, all members of the universe will be surveyed.  Doing so would yield 54 as the targeted number of respondents, representing all 50 states, the District of Columbia, Puerto Rico, Guam, and the Virgin Islands.  For the lower-bound estimate, we also assume that the survey will be one of the shorter ones envisioned under this clearance (no more than 10 separate questions).  We assume further that each question will take no more than 1 minute to answer, which we take to be ample if informed respondents are asked questions of a simple factual nature about state policy.  In this case, the aggregate hours of burden would be 9 hours.  This estimate represents the lower bound of aggregate burden for a single survey.   If eight surveys were conducted over the three years, the average annual burden would be 24 hours.

 

For surveys targeted to a universe containing more than 250 entities in the universe, stratified random sampling will be employed such that 250 entities will be selected to be surveyed.  Thus, 250 represents the greatest number of respondents to which any single survey would be administered.  For an upper-bound estimate of burden, we assume that the maximum number of questions under this clearance would be 30, and that the maximum amount of time for each question to be answered would be 3 minutes on average.  This estimate assumes that providing the answer to some of the questions might require the respondent to either consult written documents (e.g., policy issuances) or consult with colleagues.  In this case, there would be 375 hours per survey for 20 surveys or 7,500 aggregate hours.  Over three years, the average annual burden would be 2,500 hours.

 

Table 1

Upper-Bound and Lower-Bound Estimates of Total Burden Hours per Survey

 

		Sample Size		# of Questions		Avg Time per Question	Aggregate Burden Hours per survey	Estimated Number of Surveys Over 3 Yrs	Total Annual Burden Hours
Lower-Bound		54		10		1 minute	9 hours	8	72
Upper-Bound		250		30		3 minutes	375 hours	20	7,500

 

 

 

Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage and rate categories.  The cost of contracting out or paying outside parties for information collection activities should not be included here.  Instead, this cost should be included in Item 13.

The range of costs to respondents would be as follows:

Lower-Bound:  

54 respondents x 10 (10 questions @ 1 minute/question) = 9 hours x  $30 = $270 per survey; 8 surveys @ $270 = $2,160  ÷ 3 years = $720 in annualized costs to respondents

Upper-Bound:   

250 entities x 90 minutes (30 questions @ 3 minutes/question) = 375 hours x $30 =  $11,250 per one survey; 20 surveys @ $11,250 = $225,000 ÷ 3 years = $75,000 in annualized costs to respondents

The $30 per hour estimate is based on what a reasonable standard hourly wage is for the administrative staff who will be answering the survey.

13.     Provide an estimate for the total annual cost burden to respondents or recordkeepers resulting from the collection of information.  (Do not include the cost of any hour burden shown in Items 12 and 14).

 

The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life) and (b) a total operation and maintenance and purchase of services component.  The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information.  Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred.  Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and record storage facilities.

The proposed information collection plan will not require the respondents to purchase equipment or services or to establish new data retrieval mechanisms. 

(a) We do not expect any total capital and start-up costs.

(b) We do not expect extensive time spent on generating, maintaining, and disclosing or providing the information.

If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance.  The cost of purchasing or contracting out information collections services should be a part of this cost burden estimate.  In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

The proposed information collection plan will not require the respondents to purchase equipment or services or to establish new data retrieval mechanisms.  These costs are not expected to vary.

Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.

We do not expect responding agencies to purchase equipment or services in order to respond to this information collection plan effort.

 

14.     Provide estimates of annualized costs to the Federal government.  Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information.  Agencies may also aggregate cost estimates from Items 12, 13, and 14 in a single table.

The Federal government’s costs for this effort will derive from the contractors who will conduct the surveys.  To compute these costs, we assume that data collection and analysis would cost between $40,000 and $80,000 based on estimates developed by ETA’s contractor for the first survey we are developing.   These estimates take into account all expenses, including variable hours for different professionals (research director, analyst and writer, and interviewers), as well as equipment and overhead.  

The cost of conducting 8 to 20 surveys would therefore range between $320,000 to $1.6 million over three years, or $106,667 to  $533,333 annualized costs.    A detailed cost to the Federal government for each collection of data will be provided for in each survey.

15.     Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-I.

There is no change in burden.

16.     For collections of information whose results will be published, outline plans for tabulation and publication.  Address any complex analytical techniques that will be used.  Provide the time schedule for the entire project, including beginning and end dates of the collection of information, completion of report, publication dates, and other actions.

The surveys will be administered on an as-needed basis over the three-year period covered by this clearance.  Results from the data collection will be presented to ETA by the contractor through a series of briefing papers, displaying tabular presentations of results, such as the number and percentage of states or local agencies who gave each of the various responses. 

17.  If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

ETA will display the OMB control number and expiration date for any individual surveys under this clearance. 

 

18.     Explain each exception to the certification statement identified in Item 19, Certification for Paperwork Reduction Act Submissions, of OMB Form 83-I.

There are no exceptions to the certification statement.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1.       Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used.  Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample.  Indicate expected response rates for the collection as a whole.  If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Depending on the survey, the administration will be either to the universe of respondents or to a stratified random sample.  When samples are drawn, they will be of local workforce areas or One-Stop Centers.  Drawing samples using stratified random selection in such circumstances can improve the efficiency of sample estimates when the dimension on which the sample is stratified is strongly related to the research question being examined.  Given that administrative and governance structures and service design and delivery will encompass the most likely research questions being investigated under this clearance, we propose that program size (as measured by WIA funding) is the most reasonable stratification criterion.  

On this basis, Table 3 offers an example of sampling rates and strategies for drawing samples of states, local areas, and One-Stop Centers.    The sample size of 250, noted below, is a maximum, and may not be required in all cases.    A calculation of the precision and necessary sample size will be done for each survey and provided to BLS and OMB for approval. 

Table 3

 

Universe		# of Respondents		How Drawn
State agencies		All in Universe		All in universe

 

Local workforce areas (LWIAs)		250 LWIAs (of 605 in universe)		Stratified random selection.  Five strata will be defined.  After ranking LWIAs by amount of funding, each stratum will be defined such that each will make up equal proportions of total WIA funding.  Thus, the first stratum will include the largest LWIAs that make up the top 20% of aggregate WIA funding nationwide; the next stratum will include LWIAs that make up the next 20% of funding, and so on.  An equal number of LWIAs (50, or 250/5) will be selected randomly from within each stratum.
One Stop Centers		250 Centers (of approximately 3,300 in universe)		Stratified random selection of 250 LWIAs will be selected using the method described above.  One Center will be randomly selected within each of these LWIAs using a simple random sampling.

 

 This data collection has not been previously conducted.  However, we expect to receive a response rate of at least 80% for all surveys associated with this information collection plan.  This estimate is consistent with results from previous data collection efforts of similar nature and magnitude, including the Job Corps High School Diploma Initiative Survey administered by the National Office of Job Corps, which received an 82% response rate.

 

2.         Describe the procedures for the collection of information including:

 

        Statistical methodology for stratification and sample selection,

        Estimation procedure,

        Degree of accuracy needed for the purpose described in the justification,

        Unusual problems requiring specialized sampling procedures, and

        Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

For each survey to be conducted, the OMB package will give a very detailed point-by-point statement of the population(s) of interest, the population parameters (including formulas) about which the ETA wishes to make inferences, the full sample design, the data collection instrument to be used, point estimation and variance estimation methods to be used (again, including formulas), and nonresponse adjustment or imputation methods, if any, to be used in the analysis.

For a typical survey:

a) Sample Selection.  Sample selection will be carried out using the procedures described above, in answer to question 1.

b) Estimation Procedures.  The analysis will primarily make use of frequency distributions and cross-tabulations, which will provide basic information about agency opinions and service strategies.  Basic characteristics of all LWIAs are known from existing sources (e.g., their level of funding, types of persons served, type of administrative structure, location, etc.); the data can be used to suggest whether non-respondents differ in any substantial way from respondents and to develop weights for respondents in order to accurately depict the frequencies or cross-tabulations.

c) Degree of Accuracy.  For surveys directed to state agencies, sampling will not be employed.  Thus, results should be an accurate reflection of the relevant universe, subject to the constraints of reporting error, after appropriate adjustments for non-response bias are made.  For surveys directed to LWIAs and One-Stop centers, we will select a sample of at most 250 respondent entities, as described above.  Assuming a response rate of 80% would yield completed interviews for 200 entities. 

The degree of accuracy for a survey directed to LWIAs or One-Stop centers will be determined as follows.  Suppose H strata of sizes N_h, h = 1, 2, …, H, from each of which n_h units are sampled respectively by simple random sampling (srs).  Suppose the proportion of “successes” in the hth stratum is p_h.  Then the overall proportion of successes in the population is

,

where N is the total population size and S is the total number of successes in the population.

 

An unbiased estimator of p is

,

where     , s_h  being the number of successes in the sample from the hth stratum, so that  is an estimate of the number of successes in the population.

 

The variance of this estimator is given by

 

.

d) Unusual problems.  There are no unusual problems.

e) Periodic Data Collection.  Each survey will be administered only once.

3.   Describe methods to maximize response rates and to deal with issues of non-response.  The accuracy and reliability of information collected must be shown to be adequate for intended uses.  For collections based on sampling, a special justification must be provided for any collection that will not yield reliable data that can be generalized to the universe studied.

In order to maximize response rates and minimize non-responses, ETA’s contractor will attempt to contact respondents multiple times until a completed survey is obtained.  When web surveys are used primarily, the follow-up will be by telephone; an attempt to administer the survey by telephone will be attempted on the spot. 

When phone surveys are used primarily, the contractor will attempt to reach each respondent by phone multiple times.  Should this approach not yield a completed survey, telephone calls will be made to another contact person within the targeted agency.  

Basic characteristics of state agencies and of LWIAs are known from existing sources (e.g., their level of funding, types of persons served, type of administrative structure, location, etc.).  These data can be used to suggest whether non-respondents differ in any substantial way from respondents and to develop weights for respondents in order to accurately depict the frequencies or cross-tabulations.  Because the sample will be drawn using probabilistic selection methods, results will be generalized to the universe studied after making appropriate corrections for potential selection bias.

4. Describe any tests of procedures or methods to be undertaken.  Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility.  Tests must be approved if they call for answers to identical questions from 10 or more respondents.  A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

ETA’s contractor will pretest each survey with up to, but no more than, nine respondents.  The pretest will assess the clarity of content and wording of the survey, the organization and format of the questionnaire, respondent burden time, and potential sources of response error.  The pretest will be used to modify the questionnaire as appropriate.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

ETA’s contractor(s) will refine the design and carry out the sampling plan, collect the data, and prepare data tabulations and brief analyses.  The contractor that ETA uses will vary from survey to survey.  However, the contractors will be selected because their substantive background makes them very familiar with the context of the questions being asked and because of their expertise in survey design and administration and statistical methods.