VETS employment outcomes Supporting Statement - PART B _11-13-08[1]

SUPPORTING STATEMENT FOR

PAPERWORK REDUCTION ACT SUBMISSIONS

Veterans’ Employment Outcomes Study

Part B. Collection of Information Employment Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When Item 17 on the Form OMB 83-I is checked "Yes", the following documentation should be included in the Supporting Statement to the extent that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

We are conducting this survey as part of an exploratory study that seeks to gain insight into veterans who appear to have had difficulty finding employment, including their key characteristics, the job search problems they encountered, the services they received, and services they did not receive that they would have found helpful. We can estimate the size of the potential respondent population from program statistics that indicate the number who successfully enter employment and the total user population.

DOL VETS plans to purposively select five states and then draw a random sample of veterans from within those states. States selected for the study must participate in both the Wage Record Interchange System (WRIS) and the Federal Employment Data Exchange System (FEDES), and their neighboring states should participate in WRIS so that we are able to eliminate two sources of the measurement gap. WRIS accounts for those who find employment in neighboring states, and FEDES accounts for those employed by the Federal Government, both civilian and military, and the US Postal Service. We know from interviews with selected states that WRIS accounts for a 3-9 percentage point increment in a state’s Entered Employment Rate (EER), and we assume that FEDES accounts for a 4-6 percentage point increment, depending upon the extent of federal employment in the state. Five states have agreed to participate and meet the requirements of participating in WRIS and FEDES. These states, shown in the table below, include South Dakota, Washington, Ohio, New Jersey and South Carolina. The table shows data for these states on veterans who received a staff-assisted service from DVOP specialists or LVER staff, as well as the EER from the PY 2005 VETS 200c report. Assuming that the states’ DVOP specialists and LVER staff continue to serve as many veterans as were served in PY 2005 with similar EERs, the table below estimates the potential survey population of unsuccessful veteran job seekers to be about 22,500.

State	Reported EER (2005 VETS 200c)	Assumed WRIS EER Increase (percentage points)	Assumed FEDES Increase (percentage points)	Potential EER with WRIS and FEDES	Apparently Unsuccessful Veteran Job Seekers	Veterans Served in 2005 (VETS 200c)	Potential Sampling Frame	Expected Sample Size
New Jersey	50%	5%	4%	59%	41%	20,806	8,530	416
Ohio	65%	5%	4%	74%	26%	3,668	954	47
South Carolina	67%	5%	4%	76%	24%	30,521	7,325	358
South Dakota	68%	5%	4%	77%	23%	3,133	721	35
Washington	70%	5%	4%	79%	21%	23,733	4,984	244
Total						81,861	22,514	1,100

Unfortunately, not everyone who is asked responds to a survey. Further, a survey of recently unemployed veterans, especially recent transitioners, can expect to encounter difficulties in locating sampled individuals. Experience with a similar population indicates that cooperation will be high if contact is made. We estimate a response rate of 80% and a location rate of 75%.

2. Describe the procedures for the collection of information including:

! Statistical methodology for stratification and sample selection,

! Estimation procedure,

! Degree of accuracy needed for the purpose described in the justification,

! Unusual problems requiring specialized sampling procedures, and

! Any use of periodic (less frequently than annual) data collection cycles to reduce burden.

As noted previously, the results of this study will be used to inform policymakers on broad issues regarding the employment outcomes of veterans. This survey is part of an exploratory study that seeks to gain insight on the characteristics of veterans who appear to have had difficulty finding employment. It does not seek statistical estimates that can be generalized beyond the selected states with known statistical precision. To capture the variability in veteran experiences, we plan to sample approximately 1,100 Veterans across five states. Again, we are only looking to collect descriptive information such as key characteristics of these veterans and reasons why some have difficulty or fail to find jobs, learn what services were received and what veterans thought of them, and learn what services were not received and whether they were needed.

Sampling: In purposively selecting South Dakota, Washington, Ohio, New Jersey and South Carolina, we have sought to select a geographically diverse set of states for participation in the study, as well as meeting the WRIS and FEDES requirements noted in B.1. The sample is not nationally representative and cannot be used to make statistical inferences beyond the selected states. From these states, we propose to select a proportional random sample of unsuccessful Veteran job seekers, stratified by state.

We estimate a response rate, RR, of .8 and a location rate, LR, of .75. Adjusting the estimated n above then gives

n^* = n / (RR * LR) = 1,780.

Therefore, we will draw a random sample of 1,780 veterans from the sampled states, with equal sampling rates across states. From these, we expect to obtain about 1,068 responses.

Constraints: Unfortunately, there is no nationwide database of veteran job seekers. Instead, DOL VETS must compile a sampling frame of users state by state. Each state may require a separate contract, compensation for its costs, and a data agreement. Available resources prohibit gathering user data from every state.

Estimation: Responses to the survey questions will identify those who have had difficulty finding jobs or who have still not been successful finding jobs. Responses to other questions will indicate whether EER measurement methods failed to recognize some job seekers as successful.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

The survey data collection will use several approaches to getting complete responses:

• The survey’s introduction encourages the respondent to think about his/her visit to the One Stop Career Center in acknowledging eligibility for the survey.

• The research firm that administers the survey will send individuals with a mailing address on record a postcard indicating that they may be contacted regarding a survey. The postcard will contain a brief description of the survey’s purpose. It will also provide the veteran or a family member an opportunity to update the veteran’s contact information. For veterans who do not have a phone number, the postcard will contain a toll free number to call if they would like to participate in the survey.

• Individuals that cannot be reached by phone and have an email addresses as part of their contact information will be contacted via email by the research firm that administers the CATI survey. The email will contain a toll free number the individual can call to participate in the survey. The veteran may also provide a contact phone number by return email.

• If an individual cannot be reached using contact information provided by states, the research firm that administers the survey will conduct research to find the individual’s current phone number. 

• The user contact information was recorded in conjunction with receiving a DVOP or LVER service in late 2006 or in 2007. We will obtain both primary and any alternate phone contact information, as well as e-mail addresses, that states have available.

• The research firm that administers the CATI survey will make up to four attempts to contact those veterans who are sampled. The contact attempts made for each individual record will be varied across daytime/evening hours and weekdays/weekends.

• In addition to contact information, we will obtain from the states available information on characteristics such as age, gender, education level, and branch of service. This information will support an analysis of non-response bias to indicate whether non-response is systematically related to any of these characteristics. If these characteristics do not explain non-response, then we can be more confident that the responses received are representative of this population of veterans – not just of those who responded.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

We have pre-tested the survey on selected contractor staff who are veterans. No other tests are planned.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Margaret Barton, Ph.D., SRA International, Inc., 703-284-6947

Nicole Wyrembeck, Chamberlain Research Consultants, Inc., 608-246-3010

4