The WIA [WIA Section 136(b)(2)(B)] requires that states measure satisfaction for employers and participants (job seekers) of state and local agencies that provide employment and training activities. Additionally, Section 136(b)(3)(A)(I) requires state-adjusted levels of customer satisfaction performance that shall at a minimum:
be expressed in an objective, quantifiable, and measurable form; and,
show the progress of the state toward continuously improving performance.
The Department of Labor has used the ACSI since 2000. ASCI is widely used throughout government and is the method of choice to measure customer satisfaction at the state level to fulfill this requirement of WIA.
Customer satisfaction surveys are conducted quarterly and the results are reported (by states that do not have this requirement waived) electronically to ETA each year on the WIA Annual Report (Form 9091, OMB Control No. 1205-0420).
The National Quality Research Center (NQRC) at the University of Michigan’s Ross School of Business has licensed the ACSI name, including trademark, and the methodology and survey questions to the U.S. Department of Labor for use in meeting these requirements. Further, the NQRC has provided on an annual basis the weights for states’ use in calculating their own ACSI scores (see attachments).
For the submission of annual WIASRD files, this information collection request does not contain statistical methods. Record sampling will only be allowed for tabulating customer satisfaction survey results based on a simple, random sample of participants and employers served under the WIA Title I-B program.
The survey will be conducted in all state and territories covered by the act unless they have an approved waiver. Currently, 31 states have approved waivers and do not conduct the customer satisfaction survey.
There are two different surveys that must be conducted to determine customer satisfaction unless the state has an approved waiver to opt out of the customer satisfaction survey (currently, more than half of the states have an approved waiver). This requires two universes for surveying: Participants, comprised primarily of job seekers using services funded under Title I-B of WIA; and Employers, comprised of those employers being provided services from One-Stop staff.
WIA Title I-B participants who exit services are eligible to be chosen for inclusion in the random sample. WIA Title I-B participants include individuals who are either adults, dislocated workers, older youth (ages 19-21), and younger youth (ages 14-18). All individuals from all WIA formula funding streams in an exit cohort are eligible to be chosen for inclusion in the random sample.
Those eligible for surveying include employers who have received a substantial service where the service has been completed or, if it is an ongoing service, when a full segment of service has been provided (e.g., after listing an open job order, the employer has received some referrals or if no service, 30 days have elapsed after the initial request). All employers who have received a substantial service involving personal contact with One-Stop staff are eligible to be chosen for inclusion in the random sample. When an employer has received multiple services, priority will be given to the service that required the greatest expenditure of financial assistance or staff time and the survey conducted regarding their satisfaction with that service.
The survey procedures require samples to be drawn periodically for each population. Each sample is drawn by simple, random sample procedures using a random numbers table or its automated equivalent. Grantees are required to determine the appropriate sample sizes using the required response rates and the required number of completed interviews, if appropriate. The sampling percentage should be constant over time to avoid under-representing or over-representing time intervals.
Customer satisfaction surveys are conducted quarterly and reported electronically to ETA each year on the WIA Annual Report (Form 9090).
The required minimum response rate is 70 percent for the WIA participant and employer surveys. The national aggregated response rate for Program Year 2011 was 77.6%.
The data are reviewed to determine if there are special populations for which this methodology is not reliable. The data are analyzed to determine if any sub-populations are consistently under-represented in the obtained surveys. Appropriate statistical tests are used to determine if obtained sample distributions are significantly different from the distributions of sub-populations in the population from which the sample was drawn. Tests are also undertaken to determine if the satisfaction indicator can be reliably estimated with fewer than the required number of responses. This entails a study of the impact of smaller samples on the sampling error and other distribution characteristics of the obtained sample. Since the indicator requires the comparisons of a state’s obtained performance level on the customer satisfaction indicators to its estimates, as well as to obtained performance levels from other states and grantees, any consideration of reducing sample sizes is weighed against changes of effect size.
1. WIASRD Record Design
Ryan Sutter, Employment and Training Administration, DOL, (202) 693-3821
2. Customer Satisfaction Survey Design Consultants:
David VanAmberg – Managing Director, ACSI LLC, (734) 763-9061
3. Data Collectors/Analysts
States provide results of the survey in their Annual Reports to ETA or its designated representative for analytical purposes.
C. Appendices
C.1 ACSI Methodology Report
C.2 Training and Employment Guidance Letter (TEGL) on Customer Satisfaction.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Naradzay.Bonnie |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |