M3UFO_SupportingStatementB_032221

Department of Commerce

U.S. Census Bureau

OMB Information Collection Request

Manufacturers’ Unfilled Orders Survey (MA-3000)

OMB Control No. 0607-0561

Part B -- Collection of Information Employing Statistical Methods

1. Universe and Respondent Selection

The target population for the Manufacturers’ Unfilled Orders (M3UFO) Survey is defined as all domestic companies with five or more employees that have one or more manufacturing establishments classified in any of the 2017 North American Industry Classification System (NAICS) industries that make up the 42 M3 industry categories believed to maintain unfilled orders. Eight of these industry categories are each broken into two categories, a defense component and a nondefense component. Therefore, the M3UFO survey actually covers 50 M3 industry categories. These 50 industry categories encompass the following major groups in manufacturing:

• Primary Metals

• Fabricated Metal Products

• Machinery

• Computers and Electronic Products

• Electrical Equipment, Appliances, and Components

• Transportation Equipment

• Furniture and Related Products

• Miscellaneous Products

The industries selected for the M3UFO survey are those which the U.S. Census Bureau determined to maintain considerable unfilled orders. The M3UFO survey is necessary to ensure future accuracy of the unfilled orders and new orders data in the Manufacturers’ Shipments, Inventories, and Orders (M3) Survey and to determine which NAICS industries continue to maintain unfilled orders. There is a clear need for periodic benchmarking of the M3 survey estimates to accurately reflect the manufacturing universe, and the M3UFO survey provides these benchmarks for unfilled orders data.

2. Procedures for Collecting Information

The sampling frame for the M3UFO survey is constructed using information extracted from the Census Bureau’s Business Register and updated with measure-of-size (shipments) data from the Economic Census. This sample frame represents roughly 72,000 manufacturing companies located in the United States having five or more employees and with establishments classified in any of the 50 M3 industry categories described above.

The initial sampling frame for the M3UFO survey is comprised of company by M3 industry category records.  Companies with activity in multiple industry categories have multiple records in this initial frame.  Each company by industry category record represents the combined activity of all establishments classified within that industry category.

The sample for the M3UFO survey is selected using a probability-proportional-to-size (pps) sample design based on the assigned measure of size.  Sampling is controlled at the M3 industry category level and each company is assigned a probability of selection based on its respective measure of size for each industry in which it has activity.  Therefore, each company by industry category record in the initial frame is assigned a probability of selection that is commensurate with its relative importance (based on value of shipments) within the respective industry category.  Consequently, a company with activity in multiple industry categories is assigned multiple probabilities of selection.  The final probability of selection assigned for each company is the maximum of these industry-based selection probabilities.  We also impose a minimum probability rule, which results in a maximum sample weight (inverse of the minimum probability).  This minimum probability rule reduces the risk of exceedingly large sample weights that can adversely impact sample estimates and variances.  For the M3UFO survey, the minimum probability is 0.01, resulting in a maximum sample weight of 100.  There are also some companies specified by the M3UFO survey analysts to be included in the sample as predetermined certainties.  These companies are flagged up front and included in the sample with weights of 1. The sample is allocated across the 42 unique M3 industry categories that are in scope of the M3UFO survey based on industry category priorities.  Ultimately, the sample selection process for the M3UFO survey results in a final sample of 6,000 companies.

Unfilled orders estimates are produced for each of the M3UFO industry categories using a ratio estimator.  The industry categories that have both defense and nondefense components are split based on allocations obtained from actual M3 data.  Horvitz-Thompson (HT) weighted estimates of unfilled orders and value of shipments are derived for each industry category using only companies in the M3UFO sample that have reported data for both data items, provided that shipments data are greater than zero.  The sample weight assigned to each company is the inverse of its probability of selection. Ratio estimates of unfilled orders are produced for each industry category by constructing a ratio of the weighted estimate for unfilled orders to the weighted estimate for value of shipments (UO/VS) from the M3UFO survey and applying this ratio to the corresponding Annual Survey of Manufactures (ASM) or Census of Manufactures total value of shipments estimate.  Therefore, the unfilled orders estimates for each industry category produced from the M3UFO survey are derived using the following formula:

[(HT estimate of unfilled orders) / (HT estimate of value of shipments)] * ASM or Census total value of shipments estimate

Similar to the M3 monthly link-relative estimation procedure, when a large company reports an unusual ratio of unfilled orders to shipments for a given industry category and strongly influences the overall industry ratio, the company’s data are excluded from the UO/VS ratio for that industry.  However, the company’s unfilled orders data are included in the final unfilled orders estimate for the industry category.

Estimated coefficients of variation (CVs) are computed for all unfilled orders estimates, at the total manufacturing level and for all M3UFO industry categories. Sampling was controlled for each industry category under CV constraints ranging from 0.015 to 0.20, depending on industry priority. For the most recent M3UFO survey, actual estimated CVs ranged between 0.01 and 0.18 at the industry category level, and the estimated CV at the total manufacturing level was less than 0.02, so all reliability requirements were satisfied.

3. Efforts to Maximize Response

We calculate two measures of response for the M3UFO survey. The check-in rate measures the unweighted proportion of questionnaires returned, with or without data, out of the total number of questionnaires mailed. The coverage rate measures the weighted contribution in terms of measure of size (value of shipments) for active companies with reported data included in the final unfilled orders estimates relative to the total weighted measure of size for all active companies in the M3UFO survey. The Census Bureau observed lower response in 2020 because of the Coronavirus pandemic. The check-in rate was approximately 71 percent and the coverage rate was just over 73 percent in 2020. We expect the M3UFO survey to rebound in 2021 with a check-in rate of at least 75 percent and a coverage rate of at least 80 percent.

Dedicated efforts are made to maximize the check-in and coverage rates. Since 2016, a reminder letter has been used as part of the M3UFO collection strategy to reduce the number of respondents that are classified as past due as of the 30-day reporting deadline. The reminder letter is sent to non-respondents 20 days after the initial mailing of the form. Since 2019, an email reminder has been sent to respondents five days after the mailing of initial letters to inform the contact of record for each company that an initial letter was sent. An additional due date email reminder is sent seven days before the due date to non-response companies. Companies that did not respond to the survey after 30 days received an additional follow-up letter.

Forty-five days after the follow-up letter, a telephone follow-up is conducted of non-respondents for a series of four call attempts over a four-week window.

A second telephone follow-up series is conducted to contact the largest non-response companies of each M3UFO industry category seven days after the initial telephone follow-up period has been concluded. Each of these companies is contacted until a valid response is acquired by an M3UFO analyst.

After the telephone follow-up process concludes, non-respondents receive an additional email reminder each subsequent month until a valid response is obtained electronically or by an M3UFO analyst.

The M3UFO survey will continue to use this mailing strategy going forward by sending an initial form to respondents, followed by a reminder letter and follow-up letter to non-respondents. An email strategy will be used to notify respondents that the initial letter has been mailed, followed by a due date reminder and then a series of non-response notifications. A phone strategy consisting of telephone follow-up for non-respondents with a subsequent round of telephone follow-up for the largest non-respondent companies will be used going forward for the purpose of achieving and maintaining higher check-in and coverage rates.

Report forms are reviewed and evaluated for any comments and suggestions made by the respondents to improve the survey results.

4. Tests of Procedures or Methods

Periodically, the Census Bureau staff interviews survey respondents to assess our data requests and to keep abreast of the current record keeping practices and changes in technology.

5. Contacts for Statistical Aspects and Data Collection

Person responsible for statistical methodology:

Amy M. Newman-Smith,

Methodology Director for

Manufacturing, Investment and Construction Programs

Economic Statistical Methods Division

U.S. Census Bureau

(301) 763-6595

Person responsible for data collection:

Carol Aristone, Assistant Division Chief

Manufacturing, Corp. Profits and Business Dynamics

Economic Indicators Division

U.S. Census Bureau

(301) 763-7062

List of Attachments

A. Form MA-3000

B. Instructions MA-3000(INST)

C. Major Manufacturing Activities List MA-3000(I)

D. Initial Mail-out Letter

E. Reminder Letter

F. Past Due Letter

G. Centurion Instrument Screenshots

H. Title 13 Relevant Sections