OMB Supporting Statement Part B - 10-27-21

National Compensation Survey (NCS)

OMB Control Number 1220-0164

OMB Expiration Date: 5/31/2024

SUPPORTING STATEMENT FOR THE NATIONAL COMPENSATION SURVEY

OMB CONTROL NO. 1220-0164

This clearance request is classified as a revision. The sample size of the NCS is being increased due to collection non-response rates experienced during the pandemic. Detail of changes is provided in items 1b, 2a, 2c, 3a, 3c, and 4 below. Information in Part A items 12 and 15 is also changed.

Part B. Collection of Information Employing Statistical Methods

For detailed technical materials on the sample allocation, selection, and estimation methods as well as other related statistical procedures see the BLS Handbook of Methods, BLS technical reports, and ASA papers listed in the references section. The following is a brief summary of the primary statistical features of the NCS.

As described in Sections 1 – 3 of this document, the NCS sample is selected using a 2-stage stratified design with probability proportional to employment sampling at each stage. The first stage of sample selection is a probability sample of establishments within pre-defined geographic areas, industry, and ownership type (privately-owned and State and local government), and the second stage of sample selection is a probability sample of jobs within sampled establishments. The NCS uses 24 geographic areas, one for each of the 15 largest metropolitan areas by employment and one for the remainder of each of the nine Census Divisions. Data from all sampled establishments are used to produce the cost and benefit products.

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

1a. Universe

The NCS measures employee compensation in the form of wages and benefits for detailed geographic areas, industries, and occupations as well as national level estimates by industry and occupation. The universe for this survey consists of the Quarterly Contribution Reports (QCR) filed by employers subject to State Unemployment Insurance (UI) laws. The BLS receives these QCR for the Quarterly Census of Employment and Wages (QCEW) Program from the 50 States, the District of Columbia, Puerto Rico, and the U.S. Virgin Islands. The QCEW data, which are compiled for each calendar quarter, provide a comprehensive business name and address file with employment, wage, detailed geography (e.g., county), and industry information at the six-digit North American Industry Classification System (NAICS) level. This information is provided for over nine million business establishments that are required to pay for unemployment insurance, most of which are in the scope of this survey.

1b. Sample

Stratification, Sample Allocation, and Sample Selection

The NCS selects samples using a 2-stage stratified design with probability proportional to employment sampling at each stage. The first stage of sample selection is a probability sample of establishments within pre-defined geographic areas, industry, and ownership of interest, and the second stage of sample selection is a probability sample of jobs within sampled establishments. For more information on the sample design for private industry establishments, see the ASA paper by Ferguson et al. titled, “Update on the Evaluation of Sample Design Issues in the National Compensation Survey,” (https://www.bls.gov/osmr/research-papers/2011/pdf/st110230.pdf). For more information on the sample design for State and local government businesses, see the ASA paper by Ferguson et al. titled, “State and Local Government Sample Design for the National Compensation Survey,” (https://www.bls.gov/osmr/research-papers/2012/pdf/st120280.pdf).

Each sample of establishments is drawn by first stratifying the establishment sampling frame by defined geographic area of interest, industry, and ownership. The new industry strata for State and local government as well as private industry samples are shown below (North American Industry Classification System – NAICS-based). The 24 geographic areas of interest for the NCS appear in the third table below.

NCS Industry Stratification for State and Local Government Samples

Aggregate Industry	Detailed Industry	Included NAICS Codes	Establishments in Frame	Sample Size
Education	Elementary and Secondary Education	6111	61,939	631
Education	Colleges and Universities	6112, 6113	7,457	205
Education	Rest of Education	61 excl 6111-6113	1,256	6
Financial Activities	Other Service-providing - Part A	52-53	244	24
Goods Producing	Goods-Producing	21, 23, 31-33	6,349	25
Health Care, including Hospitals and Nursing Care	Hospitals	622	2,571	83
Health Care, including Hospitals and Nursing Care	Nursing Homes	623	2,100	16
Health Care, including Hospitals and Nursing Care	Rest of Health	62, excl 622-623	8,490	28
Service Providing	Trade, Transportation, and Utilities	42, 44-45, 48-49, 22	12,847	49
Service Providing	Public Administration	92 excl 928	108,358	472
Service Providing	Other Service-providing - Part B	51, 54-56, 71-72, 81 excl 814	20,430	57

NCS Industry Stratification for Private Establishment Samples

Aggregate Industry	Detailed Industry	Included NAICS Codes	Establishments in Frame	Sample Size*
Education	Elementary and Secondary Schools	6111	18,890	132
Education	Junior Colleges, Colleges and Universities	6112, 6113	11,017	401
Education	Educational Services (Rest of)	61 (excl 6111-6113)	99,250	84
Finance, Insurance and Real Estate	Finance (Rest of)	52 (excl 524)	305,097	1409
Finance, Insurance and Real Estate	Insurance	524	197,446	856
Finance, Insurance and Real Estate	Real Estate, Renting, Leasing	53	421,705	306
Goods Producing	Mining	21	31,940	114
Goods Producing	Construction	23	829,664	1357
Goods Producing	Manufacturing	31-33	354,966	1556
Health Care, including Hospitals and Nursing Care	Healthcare, Social Assistance (Rest of)	62 (excl 622, 623)	1,569,127	282
Health Care, including Hospitals and Nursing Care	Hospitals	622	10,835	365
Health Care, including Hospitals and Nursing Care	Nursing and Residential Care Facilities	623	82,571	573
Service Providing	Utilities	22	19,072	184
Service Providing	Wholesale Trade	42	610,854	1039
Service Providing	Retail Trade	44-45	1,035,320	2152
Service Providing	Transportation and Warehousing	48-49	264,161	453
Service Providing	Information	51	189,500	537
Service Providing	Professional, Scientific, Technical	54	1,277,558	600
Service Providing	Management of Companies and Enterprises	55	70,529	118
Service Providing	Admin., Support, Waste Management	56	567,981	661
Service Providing	Arts, Entertainment, Recreation	71	156,517	147
Service Providing	Accommodation and Food Services	72	726,279	597
Service Providing	Other Services (excl Public Administration)	81 (excl 814)	613,781	501

* The sample size is the estimated total three-year sample size allocated to each industry. Approximately one-third of these units will be initiated each year.

24 Geographic Areas for the NCS Sample Design

Atlanta--Athens-Clarke County--Sandy Springs, GA CSA

Boston-Worcester-Providence, MA-RI-NH-CT CSA

Chicago-Naperville, IL-IN-WI CSA

Dallas-Fort Worth, TX-OK CSA

Detroit-Warren-Ann Arbor, MI CSA

Houston-The Woodlands, TX CSA

Los Angeles-Long Beach, CA CSA

Minneapolis-St. Paul, MN-WI CSA

New York-Newark, NY-NJ-CT-PA CSA

Philadelphia-Reading-Camden, PA-NJ-DE-MD CSA

San Jose-San Francisco-Oakland, CA CSA

Seattle-Tacoma, WA CSA

Washington-Baltimore-Arlington, DC-MD-VA-WV-PA CSA

Miami-Fort Lauderdale-Port St. Lucie, FL CSA

Phoenix-Mesa-Scottsdale, AZ MSA

Rest of New England Census Division (excl. areas above)

Rest of Middle Atlantic Census Division (excl. areas above)

Rest of East South Central Census Division (excl. areas above)

Rest of South Atlantic Census Division (excl. areas above)

Rest of North Central Census Division (excl. areas above)

Rest of West North Central Census Division (excl. areas above)

Rest of West South Central Census Division (excl. areas above)

Rest of Mountain Census Division (excl. areas above)

Rest of Pacific Census Division (excl. areas above)

* The above area definitions are based on the 2010 Decennial Census data which was released by the Census Bureau in February of 2013. NCS began using the above definitions with the State and Local Government sample selected in FY2015 which began collection in June 2015. The first private sample using the above definitions was selected in FY2016 and began collection in June 2016. NCS Samples prior to FY2015 were selected using the area definitions released by the Census Bureau in 2009.

After the sample of establishments is drawn, jobs are selected in each sampled establishment. The number of jobs selected in a private establishment ranges from 4 to 8 depending on the total number of employees in the establishment, except for aircraft manufacturing establishments and establishments with fewer than 4 workers. In government entities, the number of jobs selected ranges from 4 to 20, except for establishments with fewer than 4 workers. In aircraft manufacturing, the number of jobs selected ranges from 4 for establishments with fewer than 50 workers to 32 for establishments with 10,000 or more workers, unless the establishment has fewer than 4 workers. In establishments with fewer than 4 workers, the number of jobs selected equals the number of workers. The probability of a job being selected is proportional to its employment within the establishment, except in cases where all jobs are selected from the establishment.

Scope – The NCS sample is selected from the populations as defined above.

Sample Allocation – The currently approved total NCS sample consists of approximately 11,400 establishments. BLS requests to increase the sample allocation to 16,000 establishments. The currently approved private portion of this sample is approximately 9,800 establishments. Because government response rates are at acceptable levels, BLS is requesting to increase only the private portion of the sample to approximately 14,400 establishments. The private portion is allocated based on strata defined by the survey’s 24 geographic areas and five aggregate industries. The government portion of this sample, approximately 1,600 establishments, is also allocated based on strata defined by the survey’s 24 geographic areas and five aggregate industries, although the detailed industries for governments are different than those for private industry. Self-representing, certainty establishments are assigned a sampling weight of 1.00, and non-certainty establishments are assigned a sampling weight equal to the inverse of their selection probability.

For private industry samples, NCS computes detailed allocations and identifies multi-year certainty establishments once every three years under the three-year rotation cycle. If budget or resource levels change significantly, allocation will be recomputed, multi-year certainties will be re-selected, and a new three-year rotation will begin.

The sample allocation process starts with a total budgeted sample size. The NCS will use targeted percentages across industries along with the frame data to determine how to distribute the sample units among the sampling cells for private industry samples. Due to the small sample size of the NCS for the private non-aircraft manufacturing allocation, NCS uses a five aggregate-industry stratum allocation with a modified measure of size within each of the 23 detailed industries. This adjustment to the measure of size (MOS) allows fewer strata but controls the number of units needed in the twenty-three detailed industries for which the NCS wants to publish private industry estimates.

The total three-year NCS private non-aircraft manufacturing sample size is first allocated to the five aggregate-industry strata. The sample size of each stratum is calculated so that the distribution of the new sample mirrors the desired distribution of the full sample in order to maximize the ability to meet publication goals. Next, each of the five aggregate stratum allocations is divided among the 24 geographic areas in proportion to the total adjusted employment of the frame units in the areas, resulting in 120 initial area-industry cell allocations. The MOS of a frame unit is the product of the unit’s employment and an adjustment factor that is used to maintain the current distribution of the sample among the 23 detailed NCS industries.

Multi-year private industry certainty units are identified using the initial cell allocations and the adjusted MOS. Each initial area-industry cell allocation is then reduced by the number of certainty units in the cell to create 120 non-certainty area-industry cell allocations. The MOS adjustment factors are recalculated to exclude the certainty units. Finally, the non-certainty allocations are divided among the three years of the sample design by distributing the non-certainty sample sizes across each of the three years. This distribution is done by dividing each size by three and assigning the integer portion of the result to each of the three years. The remainder is assigned to the appropriate number of years, one establishment at a time, in a manner that allows each annual selected sample to be the same size and the size for each sampling cell to vary by no more than one from year to year.

Under normal processing, private industry sample allocation and identification of multi-year private certainty units are executed once every three years. During years when prior year allocations and multi-year certainties are being used, the most recently identified set of multi-year certainty establishments are removed from the frame for operational purposes and added to the final selected sample. This ensures that each sample group represents the entire frame. The most recent non-certainty allocations and the sample frame without the multi-year certainties are used to compute the final measure of size adjustment factor and to set the non-certainty sample size for each of the 120 area-industry sampling cells.

A sample of private aircraft manufacturing establishments is selected once every three years at the same time that we begin a new three-year rotation for the rest of private industry. Measures of size are set for these establishments in the same manner as is done for the other private industry establishments, except that there is no adjustment for detailed industries within this group.

A sample of State and local government, also referred to as the public sector, establishments is selected once every ten years. The public sector allocation process starts with the total budgeted sample size. First, one establishment is allocated to each of the 120 sampling cells. The remaining total sample size is then allocated to the five aggregate industry strata in proportion to the total employment within each industry. Finally, each of the five aggregate stratum allocations is divided among the 24 geographic areas in proportion to the total employment of the frame units in the areas, resulting in 120 area-industry cell allocations.

State and local government certainty units are identified using the initial cell allocations and the establishment MOS. When an establishment MOS is greater than the total employment in the cell divided by the initial cell allocation for the cell, the public sector establishment is flagged as a certainty unit and the remaining cell allocation is reduced by the number of certainty selections. This identification process is repeated until no more certainty units exist in any cell. Each initial area-industry cell allocation is then reduced by the number of certainty units in the cell to create 120 non-certainty area-industry cell allocations.

Sample Selection – Under this design, NCS selects an independent non-certainty sample of establishments every year within each of the five aggregate industry and 24 geographic area sampling cells. Within each of the sampling cells, units are sorted by detailed industry (23 for private and 11 for public sector), final adjusted MOS, and establishment identification number. The selection process follows a systematic Probability Proportionate to Size (PPS) approach where the measure of size includes the adjustment factor as defined in the allocation section. The private industry multi-year certainty units identified in the previous step are added to each private industry non-certainty sample to form the entire establishment sample.

Sample weights are assigned to each of the selected non-certainty establishments in the sample to represent the non-certainty portion of the frame. Units selected as certainty are self-representing and will carry a sample weight of one. The sample weight for the non-certainty units is the inverse of the probability of selection.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

2a. Sample Design

Sample Rotation –All current sample groups, private, State and local government follow the national based design as described above. New Private industry samples are selected and replaced annually with certainty units and aircraft manufacturing units selected in the first year of a three year rotation. For the State and local government sectors, a full sample is selected approximately every 10 years.

All sample units, private, State and local government, are assigned to one of four collection panels for initiation. Once a sample of establishments is selected and collection panels have been assigned, BLS Regional Office employees review and refine the sample before collection begins. As part of this refinement process, establishments may be moved from one collection panel to another to coordinate initiation with more than one establishment and/or to reduce travel costs associated with initiation efforts. Establishments are initiated over a fifteen month period with one collection panel required to be completed every three months after an initial 3 month detailed refinement period. Once initiated, a unit is then updated quarterly until it rotates out of the design. No newly initiated establishment is used in the NCS estimates until the entire sample has been initiated and updated for a common/base quarter. During this base quarter, data for the newly initiated sample as well as the prior sample is updated. After the base quarter is completed, the entire sample is added to the data available for estimation while the oldest sample in estimation is dropped from further updates and inclusion in the estimates.

Sample Replacement Scheme

Each year, the NCS selects a new sample of private sector establishments from the most recent available frame data. A new sample of jobs is selected within each private establishment at least once every 3 years (10 years for government) as the establishment is initiated into the survey process. The entire private NCS sample is completely replaced over a three year period on a rotational basis.

The primary objectives of the replacement scheme are to reduce the reporting burden of individual establishments by rotating units out of the sample and to insure that the establishment sample is representative of the universe it is designed to cover over time. Because samples are selected independently from year to year, there is a chance for establishments to be selected in a future replacement sample(s) regardless of whether or not it has already rotated out of the current sample.

For Fiscal Year 2022, the NCS will not rotate out a private update sample group when the new sample group completes the initiation phase. NCS will continue to update four sample groups, rather than three, to be used in all estimation quarters through the September 2022 quarter in an attempt to rebound from the loss of data experienced as a result of the Coronavirus pandemic. This additional private sample group is anticipated to rotate out after the September 2022 quarter at which time the BLS expects to rotate out two private sample group to return to the normal three sample group design. The State and Local Government sample will remain unchanged during this period.

The NCS three-year sample design includes an estimated 9,800 private establishments. The BLS goal is for 75% of the original private sample to be used in index estimation. Future analysis will determine if the usable FY21-FY22 initiation sample will maintain a sufficient number of private units in estimation. If it will not, BLS will rotate out only the oldest sample group after the September 2022 quarter and continue to update four sample groups in Fiscal Year 2023. The requested initiation sample increase in FY22-23 is expected to yield the number of units at the lower response rate that will support standard rotation.

2b. Estimation Procedure

The National Compensation Survey (NCS) produces indexes measuring change over time in labor costs through the Employment Cost Index (ECI) and the level of average costs per hour worked through the Employer Costs for Employee Compensation (ECEC). The NCS also provides benefits incidence for the percentage of workers with access to and participating in employer-sponsored benefit plans. The survey covers a broad range of benefits including holidays and vacations, sick leave, health and life insurance, and retirement plans. Details of employer-provided health and retirement plan provisions are also available.

The ECI is a measure of the change in the employer costs of labor, independent of the influence of employment shifts among occupations and industry categories. The total compensation series include changes in wages and salaries and in employer costs for employee benefits. The ECI calculates indexes of total compensation, wages and salaries, and benefits separately for all civilian workers in the United States (as defined by the NCS), for private industry workers, and for workers in State and local government. For all of these categories, the ECI calculates the same measures by occupational group, industry group, and worker and establishment characteristics. Seasonally adjusted series are calculated as well.

The ECI is a modified Laspeyres index (i.e., index reflecting the change in labor costs over time), where the basic computational framework for the ECI is the standard formula for an index number with fixed index weights, modified by special statistical conditions and accounting for sampling methodology.

The ECEC series measures the average cost to employers for wages and salaries, and benefits, per employee hour worked. The series provides data on employer costs per hour worked for total compensation, wages and salaries, total benefits, and the following benefits: paid leave—vacations, holidays, sick leave, and personal leave; supplemental pay—premium pay (such as overtime, weekend, and holiday) for work in addition to the regular work schedule and for shift differentials, and nonproduction bonuses (such as yearend, referral, and attendance bonuses); insurance benefits—life, health, short-term disability, and long-term disability insurance; retirement and savings benefits—defined benefit and defined contribution plans; and legally required benefits—Social Security (refers to Old-Age, Survivors, and Disability Insurance (OASDI) program), Medicare, federal and state unemployment insurance, and workers’ compensation. Cost data are presented both in dollar amounts and as percentages of total compensation and published quarterly.

The NCS collects and publishes data annually on the incidence of employer-provided benefits and on the key provisions (terms) of employee benefit plans, for civilian workers, workers in private industry, and State and local government workers. In addition, the NCS publishes data on detailed provisions of coverage in two major benefit areas: health insurance and retirement plans. Health data include medical plan provisions, such as deductibles, coinsurance, and out-of-pocket maximums, as well as details of dental, vision, and prescription drug benefits. Provisions of defined benefit and defined contribution retirement plans, such as eligibility requirements and benefit formulas, also are published. Detailed provision estimates are produced based on the initiation year (e.g., first year of participation in the NCS for the sampled establishment) of each sample group collected data via Summary Plan Description (SPD), plan summary sheets, and Summary of Benefits and Coverage (SBC).

For more detailed information on the procedures used in the calculation of the NCS estimates see the BLS Handbook of Methods (available on the BLS Internet at https://www.bls.gov/opub/hom/ncs/home.htm).

2c. Reliability

The estimation of sample variances for the NCS is accomplished through the method of Balanced Repeated Replication (BRR). This replication technique uses half-samples of the original sample and calculates estimates using those sub-samples. The replicate weights in both half-samples are modified using Fay’s method of perturbation. The sample variance is calculated by measuring the variability of the estimates across these sub-samples. For a mathematical presentation of this method, see the BLS Handbook of Methods listed in the references.

Measuring the Quality of the Estimates

The two basic sources of error in the estimates are bias and variance. Bias is the amount by which estimates systematically do not reflect the characteristics of the entire population. Many of the components of bias can be categorized as either response or non-response bias.

Response bias occurs when respondents’ answers systematically differ in the same direction from the correct values. For example, this occurs when respondents incorrectly indicate no change in benefits costs when benefits costs actually increased. Another possibility of having response bias is when data are collected for a unit other than the sampled unit. Response bias can be measured by using a re-interview survey. If properly designed and implemented, re-interview surveys can also indicate where improvements are needed and how to make these improvements. The NCS has a Technical Re-interview Program (TRP) that does a records check of a sample of each field economist’s schedules of collected data. TRP is a part of the overall review process. TRP verifies directly with respondents a sample of elements originally collected by the field economist. The results are reviewed for adherence to NCS collection procedures. Although not explicitly used to measure bias, this program allows the NCS to identify procedures that are being misunderstood and to make improvements in the NCS Data Collection Manual and training program.

Non-response bias is the amount by which estimates obtained do not properly reflect the characteristics of non-respondents. This bias occurs when non-responding establishments have earnings and benefit levels and changes in earnings and benefits that are different from those of responding establishments. Non-response bias is being addressed by continuous efforts to reduce the amount of non-response. The results from an initial analysis were documented in the 2006 ASA Proceedings of Survey Research Methods Section.¹ A follow-up study from 2008 is also listed in the references. NCS continues to monitor response rates to ensure that the potential for non-response is kept as low as possible. Details regarding adjustment for non-response and current non-response bias research are provided in Section 3 below.

Another source of error in the estimates is sampling variance. Sampling variance is a measure of the fluctuation between estimates from different hypothetical samples using the same sample design. Sampling variance in the NCS is calculated using a technique called balanced half-sample replication. For national estimates this is done by forming 120 different re-groupings of half of the sample units. For each half-sample, a "replicate" estimate is computed with the same formula as the regular or "full-sample" estimate, except that the final weights are adjusted. If a unit is in the half-sample, its weight is multiplied by (2-k); if not, its weight is multiplied by k. For all NCS publications, k = 0.5, so the multipliers are 1.5 and 0.5. Sampling variance computed using this approach is the sum of the squared difference between each replicate estimate and the full sample estimate averaged over the number of replicates and adjusted by the factor of 1/(1-k)² to account for the adjustment to the final weights. For more details, see the BLS Handbook of Methods. Standard error, which is the square root of variance, for primary aggregate estimates of the index of quarterly change is typically less than 0.5 percent. Relative standard error, which is the square root of variance divided by the estimate, for aggregate estimates of compensation, wage, or benefit levels are typically less than 5 percent. The standard errors or relative standard errors are included within published NCS reports at the following website: http://www.bls.gov/ncs/ect/ectvar.htm.

Variance estimation also serves another purpose. It identifies industries and occupations that contribute substantial portions of the sampling variance. Allocating more sample units to these domains often improves the efficiency of the sample. These variances will be considered in allocation and selection of the future replacement samples.

While declining response rates impact the data quality, the recent declines in response rates have not increased the variance estimates. Variances are calculated from all usable data (collected and imputed). The declining response rates have increased the imputation rates as missing values (nonresponse) are replaced with usable collected data. As imputation rates increase, fewer observations will be available to serve as donors or contribute to cell-means imputation. Increased imputation rates have a negative impact on estimate quality which cannot be discerned in the estimate variances.

2d. Data Collection Cycles

NCS data are collected quarterly for all schedules.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

There are three types of non-response: permanent non-response, temporary non-response, and partial non-response. The non-responses can occur at the establishment level, occupation level, or benefit item level. The assumption for all non-response adjustments is that non-respondents are similar to respondents, within certain cells formed by cross-classifying establishment characteristics.

To adjust for permanent establishment or occupation non-response at the initial interview, weights of responding units or occupations that are deemed to be similar are adjusted appropriately. Establishments are considered similar if they are in the same ownership, 2-digit NAICS, and establishment employment size class. If there are no sufficient data at this level, then a broader level of aggregation is considered.

For temporary and partial non-response, a replacement value is imputed based on information provided by establishments with similar characteristics. Imputation is done separately for each benefit both in the initial period and in subsequent update periods. Imputation is also done for each missing wage estimate after the initial period. In the rare event that the BLS cannot determine whether or not a benefit practice exists for a non-respondent, the average cost is imputed based on data from all responding establishments (including those with no plans and plans with zero costs).

There is a continuous effort to maximize response rates. We are developing and providing respondents with new and useful products to demonstrate the importance of NCS data and their participation. Examples include the Beyond the Numbers publications (http://www.bls.gov/opub/btn/) as well as videos and infographics (https://www.bls.gov/eci/videos.htm). We are continually exploring alternative methods for respondents to report their data.

For March 2020 quarter, the un-weighted response rates for NCS initiation establishments was 74% and 81% during update collection. The weighted response rates for NCS was 72% and 78% respectively. Since March 2020, the Coronavirus pandemic has significantly impacted the initiation collection. For December 2020, the unweighted response rates for NCS initiation establishments was 55%. 

3a. Maximize Response Rates

To maximize the response rate for this survey, field economists initially refine addresses ensuring appropriate contact with the employer. Then, employers are mailed a letter explaining the importance of the survey and the need for voluntary cooperation. The letter also includes the Bureau’s pledge of confidentiality. A field economist calls the establishment after the package is sent and attempts to enroll it into the survey. Non-respondents and establishments that are reluctant to participate may be re-contacted by additional field staff, such as those specially trained in refusal aversion and conversion. Additionally, respondents are offered a variety of methods, including telephone, fax, email, personal visit, and the internet, through which they can provide data. As a result of collection under the COVID-19 pandemic, the BLS recently added video collection as an option for the respondents to provide data.

It is important to note the requested sample increase is not expected to change the response rates. Response rates have been trending down for many years across government surveys. The sample increase is expected only to maintain a sufficient level of data for estimation.

3b. Non-Response Adjustment

As with other surveys, despite all efforts to maximize response rates NCS experiences non-response. To adjust for the non-responses, NCS has divided the non-response into two groups, 1) unit non-respondents and 2) item non-respondents. Unit non-respondents are the establishments or jobs that do not report any compensation data, whereas item non-respondents are the establishments that report only a portion of the requested compensation data, such as wages for a sub-set of sampled jobs.

Unit non-response is treated using a Non-Response Adjustment Factor (NRAF). Item non-response is addressed by filling in missing data with plausible values had the data been actually collected, i.e. by item imputation. Within each sampling cell, NRAFs are calculated each year based on the ratio of the number of viable establishments to the number of usable respondents in that month. The details regarding the NRAF procedure are given in the Bureau of Labor Statistics’ Handbook of Methods (https://www.bls.gov/opub/hom/ncs/home.htm).

The method used to adjust for item non-response at the establishment and quote (sampled occupation) level is a cell-mean-weighted procedure. Details of this procedure are available in the BLS Handbook of Methods (https://www.bls.gov/opub/hom/ncs/home.htm). Other techniques are used to impute for item non-response for benefit estimates and are described in the following CWC article: “Recent Modifications of Imputation Methods for National Compensation Survey Benefits Data,” which may be found at the following link: http://www.bls.gov/opub/mlr/cwc/recent-modification-of-imputation-methods-for-national-compensation-survey-benefits-data.pdf.

3c. Non-Response Bias Research

Research has been completed in the past to assess whether the non-respondents to the NCS survey differ systematically in some important respect from the respondents of the survey and would thus bias NCS estimates. Details of this study are described in the two papers by Ponikowski, McNulty, and Crockett referenced in Section 6. The NCS planned to begin a formal non-response bias analysis in the summer of 2021. However, the immediate work to assess potential sample increases to mitigate the large pandemic non-response issue took precedence. The basic assessment of current response does not indicate a shift in pattern within the non-response. All areas, industries, and size classes with private ownership have shown similar sharp decreases. A formal analysis is expected to begin in FY2022.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

4a. Tests of Collection Procedures

The NCS identified two projects to explore alternative methods of collection for data elements that are currently being collected from respondents. The major goal of the first project is for a reduction in burden for private sector respondents. Prior to this study, respondents were asked for their expenditures for State Unemployment Insurance (SUI), in order to calculate SUI costs per hour worked. The Quarterly Census of Employment and Wages (QCEW) collects, with a lag, information on an establishment’s quarterly contributions to SUI, along with taxable wages paid quarterly for SUI purposes. The ratio of the two can be used, in combination with other information already collected from the respondent, to form an alternative measure of SUI costs per hour worked. Analysis is now complete and this alternative collection is in production and used instead of direct collection.

The second project under the NCS is a test to perform text analysis on a sample Summary of Benefits and Coverages (SBCs) collected from NCS respondents. Production of SBCs is required by the Affordable Care Act (ACA). In order to facilitate comparisons between plans, SBCs use a standardized template, which may be customized by respondents to capture plan particularities. This makes them more readily amenable to automated analysis through text analysis techniques. Three phases of this work have been completed. (1) A text analysis program was created to demonstrate the feasibility of using computer assisted collection for SBCs. (2) Text analysis was used to successfully code deductible dollar values for out-of-pocket maximum and overall deductible statements as belonging, or not, to four categories: in-network individual, in-network family, out-of-network individual, and out-of-network family. (3) Workers on Amazon’s Mechanical Turk service read and coded 3,000 SBCs, which were used to train and evaluate machine learning models to predict copays, coinsurance, whether copay/coinsurance applies after the deductible is paid, what unit of measure (if any) does the copay/coinsurance apply to (e.g., per visit, per test), whether the SBC explicitly states that a medical service is not covered, and whether the SBC states that there is no charge for a medical service. The goal is to continue working in FY21 to increase the accuracy of the text analysis and expand the number of provisions extracted.

4b. Tests of Survey Design Procedures

The NCS has revised its sample design over the years, and, in February 2011, the Bureau of Labor Statistics began implementing a change to the Locality Pay Survey (LPS) component of the NCS which is used to produce annual occupational earnings data for the nation, each Census Division, and selected geographic areas. Under this change occupational earnings estimates are now produced using a modeling technique that combines the national data from the NCS with the locality data from the Occupational Employment Statistics (OES) survey (BLS Handbook of Methods, Chapter 3). All other data estimates computed using NCS data, including the Employment Cost Index, Employer Costs for Employee Compensation, and various measures of access and participation in employer-provided benefits continue to be produced. With the elimination of a need to produce locality estimates directly from the NCS sample, the remaining NCS outputs could follow a more efficient national-based sample design.

The new design, now in full production, is a national design with a 3-year sample rotation cycle. Prior to the first production sample, NCS completed the evaluation and testing for this change as detailed in this section.

The BLS staff examined potential changes to the NCS sample design that included the following options:

• Moving from an area-based sample design to a national design, thus eliminating the first stage of sampling to select areas

• Implementing a new allocation methodology to correspond with the non-area-based sampling

• Moving from a five-year rotation to a three-year rotation for private industry establishments

For each of these options, NCS tested the proposed change using the general scheme described below.

• Obtain a full frame of data,

• Use establishment total wage data from the frame to compute average monthly wages across all establishments,

• Implement the proposed change using the full frame of data,

• Select multiple (100 or more) simulated samples using the proposed methodology,

• Compute estimates of the average monthly wages using the weighted data from each of the simulated samples,

• Compute the mean and standard error of the average monthly wages across all the simulated samples, and

• Compare the estimated average monthly wages across the simulated samples to those from the frame.

In addition to analyzing the potential effect of the redesign on the reliability of the estimates, the effect of any redesign on response rates and bias was studied with details described in the paper, "Update on the Evaluation of Sample Design Issues in the National Compensation Survey," by Ferguson et al.

The NCS also evaluated options for the transition to a national design for State and local government samples. This research and evaluation were done using similar methods to those used for the private industry sample design evaluation. This research is documented in the paper “State and Local Government Sample Design for the National Compensation Survey” by Ferguson et al.

The sample design for State and local government was tested using sample simulations for which we obtained a complete sample frame from the second quarter of 2011, assigned measures of size, executed the allocation process, and selected certainty establishments. We then selected 100 non-certainty samples and evaluated the resulting samples to ensure that the total weighted employment for the samples matched the frame employment and that the desired sample sizes were obtained.

With the December 2016 ECI release, the NCS began producing outputs (ECI, ECEC, and incidence and key provisions) with a total sample size of approximately 11,400 establishments (Private, State and local government) sampled and collected under the new national sample design. The number of detailed estimates for these products has not changed and the measures of reliability are comparable to estimates produced under the previous locality-based design. Estimates in the NCS detailed benefits product line are produced from the most recent single initiation sample. Despite budget cuts resulting in an overall sample reduction, with the move to a three-year rotation, each initiation sample is larger than the previous five-year rotation would have yielded resulting in estimates that are of the same quality and level of detail as prior publications for this product.

Sample Size Increase in FY2022

Starting with the sample to be selected in FY2022 with initiation collection beginning in FY2022 and continuing through July 2023, the NCS will use an increased sample size of approximately 16,000 establishments. The sample will include 14,400 with private industry ownership and 1,600 with state and local government ownership. This increase in sample size for the private industry is to account for the significant decline in response rates experienced as a result of the ongoing worldwide pandemic. The NCS will continuously monitor the response rates and evaluate the sample size on an annual basis to determine if the increased sample size is still needed to reach the desired estimation goals. Additionally, the NCS will conduct research to determine if there is evidence of nonresponse bias as a result of the decline in cooperation from its respondents.

Testing of Modeled Wage Estimates

NCS implemented a model-based estimation approach to continue to produce data for the President’s Pay Agent that was previously available through the Locality Pay Survey. This model-based estimation approach allows BLS to continue to produce wage estimates by worker characteristic such as full-time vs. part-time or union vs. non-union.

NCS completed the evaluation and test phases for producing estimates of worker wages by worker characteristics using data from both the NCS and the Occupational Employment Survey (OES). This data is now in production using a wage model that combines the large quantity of wage data available from OES along with the detailed worker characteristics from the NCS. The model is described in the August 2013 Monthly Labor Review article titled “Wage estimates by job characteristic: NCS and OES program data” by Lettau and Zamora. In a follow-up article titled “Revisiting the dilemma of review for modeled wage estimates by job characteristic” by Lettau and Zamora, the estimates obtained from the model and the criteria for publication of an estimate are reviewed. This second article also reports a more extensive set of experimental wage estimates by area, occupation, and job characteristic. In May 2020, a Beyond the Numbers article “Comparisons of hourly wage estimates by location, work levels, hours, and other factors” by Zook and Issan provides more information on the benefits of the BLS modeled wage estimates that have been published annually since 2016. This article shows ways in which modeled wage estimates can be used to compare wages by location, hours worked, pay method, and work level.

Evaluating the Chain-link Approach to Estimates

The Employment Cost Index is currently calculated using a modified Laspeyres method, where employment weights are fixed over a period (usually about 10 years) and updated to reflect compositional changes in the economy. This allows users to compare the change in compensation without the influence of changes in the composition of employment across industries and occupations. Estimates compared across changes in fixed weights are still subject to differences in the composition of employment across the national economy. The National Compensation Survey (NCS) program is evaluating adopting a chain-link approach, which is like the process it uses when updating the fixed employment weights. In fiscal year 2021, the NCS program identified the following potential benefits to implementing the chain-link approach:

1. provide a more straightforward method for index computation,

2. facilitate changes to compensation definitions without needing to wait for a reweight period,

3. afford near seamless system production process when the index does undergo reweighting,

4. enable a more straightforward method to implement sample design changes and produce corresponding standard errors, and

5. provide flexibility to handle embedded errors in the ECI cost weights.

In fiscal year 2021, the NCS program produced requirements and coded an estimation module to calculate the chain-link estimates and standard errors. In the first quarter of fiscal year 2022, the NCS program will incorporate a module into the Employment Cost Index production environment to produce estimates and standard errors using a chain-link index formula. In the 2nd and 3rd quarters, the program staff will analyze differences between the current methodology and chain-link method estimates and standard errors. The NCS program will produce an article describing the benefit of implementing a chain-link approach and summarize the differences in the estimates and standard errors between the two methodologies. Once the analysis has been completed, the NCS program will propose a timeline for implementation, in accordance with the guidance provided by OMB. Since the NCS program is incorporating the chain-link approach to the production process, the NCS program can continue assessing the differences between the two methods as well as through the upcoming reweight, which will be completed with the estimation of December 2022 estimates. The NCS program will also update the Handbook of Methods to incorporate the methodology used to calculate chain-linked indexes along with the published estimates.

In addition to the evaluation of the chain-link approach, the NCS program has developed a process to correct prior quarter errors. This will allow the program to drastically reduce the number of microdata suppressions, which is the process currently used. Suppressing microdata increases imputation rates and reduces the available donor pool. While other BLS programs, maintain separate microdata versions that allow them to update microdata from prior periods, the ECI does not have this ability. The NCS program determined that implementing a similar version control method would require significant system resources to implement changes to the collection and estimation systems. To correct prior quarter errors, the NCS program recommended using the current period compensation costs to derive a cost for a prior (incorrect) period. The method allows for three methods to derive prior period costs by:

1. using the imputed growth rates for wages and non-wage related benefit costs,

2. using wage growth rates for wage-related benefit costs, and

3. using current quarter costs.

A benefit of implementing this approach is that the ECI estimation system already calculates the current benefit cost, imputed rate of changes, and wage change, which facilitates implementing this approach in production. The NCS program wrote requirements in fiscal year 2021 and expects to complete coding and testing in the first quarter of fiscal year 2022. The NCS program expects to have the ability to implement the derived costs process in the second quarter of fiscal year 2022, with implementation by the end of fiscal year 2022.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will collect and/or analyze the information for the agency.

Xingyou Zhang, Division Chief, Statistical Methods Group of the Office of Compensation and Working Conditions, is responsible for the statistical aspects of the NCS program. Xingyou Zhang can be reached on 202-691-6082. BLS seeks consultation with other outside experts on an as-needed basis.

References

Ferguson, Gwyn, Coleman, Joan L., Ponikowski, Chester, (August 2011), “Update on the Evaluation of Sample Design Issues in the National Compensation Survey,” ASA Papers and Proceedings, https://www.bls.gov/osmr/research-papers/2011/pdf/st110230.pdf

Ferguson, Gwyn R., Ponikowski, Chester H., McNulty, Erin, and Coleman, Joan L., (October 2012), “State and Local Government Sample Design for the National Compensation Survey”, ASA Papers and Proceedings, https://www.bls.gov/osmr/research-papers/2012/pdf/st120280.pdf

Additional References:

Crockett, Jackson, McNulty, Erin, Ponikowski, Chester H., (October 2008), “Update on Use of Administrative Data to Explore Effect of Establishment Non-response Adjustment on the National Compensation Survey Estimates,” ASA Papers and Proceedings, https://www.bls.gov/osmr/research-papers/2008/pdf/st080190.pdf

Cochran, William, G., (1977), Sampling Techniques 3^rd Ed., New York, Wiley and Sons, 98, 259-261.

Federal Committee on Statistical Methodology, Subcommittee on Disclosure Limitation Methodology, "Statistical Policy Working Paper 22," http://www.fcsm.gov/working-papers/SPWP22_rev.pdf

Ferguson, Gwyn, Ponikowski, Chester, Coleman, Joan, (October 2010), “Evaluating Sample Design Issues in the National Compensation Survey,” ASA Papers and Proceedings, https://www.bls.gov/osmr/research-papers/2010/pdf/st100220.pdf

Lettau, Michael K. and Zamora, Dee A., Monthly Labor Review (August 2013), “Wage estimates by job characteristic: NCS and OES program data,” https://www.bls.gov/opub/mlr/2013/article/lettau-zamora.htm

Myers, Michelle V. and Zamora, Dee A., Monthly Labor Review (September 2015), “Revisiting the dilemma of review for modeled wage estimates by job characteristic,” https://www.bls.gov/opub/mlr/2015/article/revisiting-the-dilemma-of-review-for-modeled-wage-estimates-by-job-characteristic.htm

Ponikowski, Chester H., and McNulty, Erin E., (December 2006), “Use of Administrative Data to Explore Effect of Establishment Nonresponse Adjustment on the National Compensation Survey Estimates,” ASA Papers and Proceedings, http://www.asasrms.org/Proceedings/y2006/Files/JSM2006-000979.pdf

Stafira, Sarah, (August 2009), “Recent Modification of Imputation Methods for National Compensation Survey Benefits Data,” U.S. Bureau of Labor Statistics, Compensation and Working Conditions Online, https://www.bls.gov/opub/mlr/cwc/recent-modification-of-imputation-methods-for-national-compensation-survey-benefits-data.pdf

US Bureau of Labor Statistics, “BLS Handbook of Methods: National Compensation Measures,” https://www.bls.gov/opub/hom/ncs/home.htm

US Bureau of Labor Statistics, “National Compensation Survey: Guide for Evaluating Your Firm’s Jobs and Pay,” https://www.bls.gov/ncs/ocs/sp/ncbr0004.pdf

Zook, David and Issan, Adam, Beyond the Numbers (May 2020), “Comparisons of hourly wage estimates by location, work levels, hours, and other factors,” https://www.bls.gov/opub/btn/volume-9/comparison-of-hourly-wage-estimates.htm

1 Ponikowski, Chester H. and McNulty, Erin E., "Use of Administrative Data to Explore Effect of Establishment Nonresponse Adjustment on the National Compensation Survey", 2006 Proceedings of the American Statistical Association, Section on Survey Methods Research [CD-ROM], American Statistical Association, 2006 http://www.asasrms.org/Proceedings/y2006/Files/JSM2006-000979.pdf

Page 18 of 18