Note to Reviewer - ORS Alternative Modes Test

February 18, 2014

MEMORANDUM FOR: REVIEWER of 1220-0164

FROM: Philip Doyle

Assistant Commissioner

Office of Compensation Levels and Trends

Bureau of Labor Statistics

SUBJECT: Non-substantive Change Request for the
National Compensation Survey (NCS) –
Alternative Modes Feasibility Test

Clearance is being sought by the Bureau of Labor Statistics’ (BLS) National Compensation Survey (NCS) program in cooperation with the Social Security Administration (SSA) for the Alternative Modes feasibility test and its associated documents. Information on this testing was included in Part A, Section 12 of the most recent Office of Management and Budget (OMB) Clearance which was approved on May 1, 2012. An earlier Non-substantive Change Request approved on September 12, 2013 included the burden hours and an explanation of this feasibility test. This request includes additional materials for the FY 2014 Alternative Modes feasibility testing which were not available at the time of the previous submission.

No additional respondent burden hours or establishments are being requested in this request. The hours were included in the NCS Non-substantive Change Request for the National Compensation Survey (NCS) for FY 2014 ORS feasibility testing.

This request seeks approval of the collection methods and documents for an Alternative Modes ORS feasibility test of 180 establishments. The previous Non-substantive Change Request for FY 2014 Feasibility Testing NCS stated the purpose of this test is to:

• Design, develop, and conduct a test to determine how best to collect occupational requirements data elements when a personal visit is not optimal due to respondent resistance, collection costs, or other factors;

The goals of the Alternative Modes feasibility test are:

1. To determine how to collect high quality ORS data via phone, e-mail and/or fax

2. To evaluate the quality of data obtained via personal visit (PV) versus data obtained without a personal visit.

The evaluation of the data quality for the Alternative Modes Feasibility Test 180 establishments will be by comparison of the Personal Visit and Alternative Collection data clusters. The samples will be evenly divided between Personal Visit collection and Alternative Collection (telephone, email, fax, etc.).  90 sample clusters were created based on similar NAICS industries, size class, and ownership in geographic regions.  One unit from each cluster will be collected via personal visit and another unit will be collected without physically visiting the establishment.  The selection of four occupations within the establishments will focus on heavily populated industry occupations to yield maximum overlap.

Due to the small number of observations, no statistical comparisons are planned.  Data quality is defined as presence of a response.

To determine how to collect high quality ORS data without a personal visit, NCS staff will hold structured internal debriefings with the field economists to discuss methods used and their perceived effectiveness in collection.

To identify possible differences in the quality of data collected, the following qualitative data comparisons are planned:

• Count the number of establishments that refuse collection, by method of collection requested: This will be compared to the number of overall units collected and the standard 30 % ref rate.

• Comparison of the number of useable occupations within collection method:  This will be a straight comparison of the percentage obtained via both methods and adjusted for small establishments where less than 4 unique occupations exist.

• Comparison of missing data (non-response) within similar occupations: This will compare the missing data by element based on collection method.

• Comparison of the number and types of outliers remaining within similar occupations:  Compare the number of data observations that fall outside of the range expected for an element based on collection method.  Range criteria are based on O*Net published data and prior ORS collection experience.

If you have any questions about this non-substantive change request, please contact Paul Carney at 202-691-5180 or e-mail at Carney_P@bls.gov or Philip Doyle at 202-691-6200 or e-mail at Doyle_P@bls.gov.

Attachments

Attachment 1 – NCS Joint Collection Phase 2 Checklist Strategy Collection Tool

Attachment 2 – Advanced MQ Collection Tool

Attachment 3 – Alternative Modes Executive Summary Email

Attachment 4 – Alternative Modes Introductory Email

Attachment 5 _ Alternative Modes Respondent Debriefing Checklist

Attachment 6 – ORS Flyer February 2014

Attachment 7 – ORS Letter February 2014

Attachment 8 – Respondent Aid

Attachment 9 – Establishment Info Sheet

Attachment 10 – Quote Info SVP and Leveling

Attachment 11 – Signed Endorsement Letter from SS

Attachment 12 – ORS Letter FY14 Chicago Public

Attachment 13 – ORS Letter FY14 Chicago Private

3