Supporting Statement B FINAL

U.S. Department of Commerce

U.S. Census Bureau

OMB Information Collection Request

Monthly Retail Surveys (MRS)

OMB Control Number 0607-0717

PART B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Universe and Respondent Selection

The Advance Monthly Retail Trade Survey (MARTS) is a subsample of approximately 4,700 units (companies and EINs) selected from the larger Monthly Retail Trade Survey (MRTS) sample of about 12,000 units.  

Sample Revision: A sample revision is the process used to re-design and re-select the samples for many of the Census Bureau’s surveys of the retail, wholesale, and service industries. We do a sample revision approximately every 5-7 years for MRTS and every 2 and one-half to 3 years for MARTS. This process is performed to:

• ensure each sample is representative of its target population

• improve the efficiency of each sample

• incorporate updates to the industry classification structure

• expand industry coverage of the survey

• update questions and instructions to obtain more accurate data

• redistribute burden for small and medium size businesses 

The current sample of retailers was introduced for April 2013 for MRTS and for November 2015 for MARTS. Our next sample will be introduced for April 2018 for MRTS and May 2018 for MARTS.  The following sections describe the methodology of the current sample.

MRTS

Sampling Frame: The sampling frame used for the MRTS has two types of sampling units: Employer Identification Numbers (EINs) and large, multiple-establishment firms. Both sampling units represent clusters of one or more establishments owned or controlled by the same firm. The information used to create these sampling units were extracted from data collected as part of the 2007 Economic Census and from establishment records contained in the Census Bureau's Business Register as updated to December 2010.

To create the sampling frame, we extract the records for all employer establishments located in the United States and classified in the Retail Trade sector as defined by the 2007 NAICS. For these establishments we extract sales, payroll, employment, name and address information, as well as primary identifiers and, for establishments owned by multi-unit firms, associated EINs. To create the sampling units for multi-unit firms, we aggregate the economic data of the establishments owned by these firms to an EIN level by tabulating the establishment data for all retail establishments associated with the same EIN. Similarly, we aggregate the data to a multi-unit firm level by tabulating the establishment data for all retail establishments associated with the same firm identifier. No aggregation is necessary to put single-unit establishment information on an EIN basis or a firm basis. Thus, the sampling units created for single-unit firms simultaneously represent establishment, EIN, and firm information.

Stratification: The primary stratification of the sampling frame is by industry group based on the detail required for publication. We further stratify the sampling units within industry group by a measure of size (substratify) into 4, 7, 10, or 13 substrata related to their annual sales. Sampling units expected to have a large effect on the precision of the estimates are selected "with certainty." This means they are sure to be selected and will represent only themselves (i.e., have a selection probability of 1 and a sampling weight of 1). Within each industry stratum, we determine a substratum boundary (or cutoff) that divides the certainty units from the non-certainty units. We base these cutoffs on a statistical analysis of data from the 2007 Economic Census. Accordingly, these values are on a 2007 sales basis. We also used this analysis to determine the number of size substrata for each industry stratum and to set preliminary sampling rates needed to achieve specified sampling variability constraints on sales estimates for different industry groups. The size substrata and sampling rates are later updated through analysis of the sampling frame.

Sample Selection: The first step in the sample selection is to identify firms selected with certainty. If a firm's annual sales or end-of-year inventories were greater than the corresponding certainty cutoff, that firm was selected into the MRTS sample with certainty.

All firms not selected with certainty were subjected to sampling on an EIN basis. If a firm had more than one EIN, we treated each of its EINs as a separate sampling unit. To be eligible for the initial sampling, an EIN had to have nonzero payroll in 2009. The EINs were stratified according to their major industry and their estimated sales (on a 2007 basis). Within each non-certainty stratum, a simple random sample of EINs was selected without replacement.

Sample Maintenance: We update the sample to represent EINs issued since the initial sample selection. These new EINs, called births, are EINs, recently assigned by the Internal Revenue Service, that have an active payroll filing requirement on the IRS Business Master File (BMF). An active payroll filing requirement indicates that the EIN is required to file payroll for the next quarterly period. The Social Security Administration attempts to assign industry classification to each new EIN.

The Census Bureau considers EINs with an active payroll filing requirement on the IRS BMF to be “BMF active” and EINs with an inactive payroll filing requirement to be “BMF inactive.”

We sample EIN births on a quarterly basis using a two-phase selection procedure [OMB No. 0607-0189]. To be eligible for selection, a birth must either have no industry classification or be classified in an industry within the scope of the Service Annual Survey, the Annual Wholesale Trade Survey, or the Annual Retail Trade Survey, and it must meet certain criteria regarding its quarterly payroll. In the first phase, we stratify births by broad industry groups and a measure of size based on quarterly payroll. A relatively large sample is selected and canvassed to obtain a more reliable measure of size, consisting of receipts in two recent months and a new or more detailed industry classification code. Births that have not returned their questionnaire after 30 days are contacted by telephone.

Using this more reliable information, in the second phase we subject the selected births from the first phase to probability proportional-to-size sampling with overall probabilities equivalent to those used in selecting the initial MRTS sample from the December 2010 Business Register. Because of the time it takes for a new employer firm to acquire an EIN from the IRS, and because of the time needed to accomplish the two-phase birth-selection procedure, we add births to the sample approximately nine months after they begin operation.

If a firm was selected with certainty and had more than one establishment at the time of sampling, any new establishments that the firm acquires, even if under new or different EINs, are included in the sample with certainty.

However, if a firm was selected with certainty and had only one establishment at the time of sampling, only future establishments associated with that firm’s originally-selected EIN are included in the sample with certainty; any new EINs that might later be associated with that firm are subjected to sampling through the quarterly birth-selection procedure.

To be eligible for the sample canvass and tabulation, an EIN selected in the non-certainty sampling operations must meet both of the following requirements:

• It must have an active payroll filing requirement on the IRS Business Master File.

• It must have been selected from the Business Register in either the initial sampling or during the quarterly birth-selection procedure.

Each quarter, we check against the current Business Register to determine if any EINs on the survey have become BMF inactive. Typically, we do not canvass BMF inactive EINs during the reference month. Likewise, if any EIN on the survey was BMF inactive in a previous reference month and is now BMF active on the current Business Register, we again include these EINs in the canvass.

Single-unit EINs selected into the sample with certainty are not dropped from canvass and tabulation if they are no longer BMF active. Rather, the firm that used the EIN is contacted, and if a successor EIN is found, it is added to the survey. For both inactive and any previously inactive EINs that are now active, data are tabulated for only the portion of the reference year that these EINs reported payroll to the IRS.

Because births are not represented in the monthly survey until they go through the two-phase selection procedure, an interim procedure is used to account for births during the period of time between the onset of activity and the time of birth selection. This interim procedure consists of imputing data for all EINs currently in the monthly survey that go out of business but are still on the IRS BMF.

Births are added to the monthly survey in February, May, August, and November of each year. At the same time, inactive EINs are removed from the survey. To minimize the effect of births and inactive EINs on the month-to- month change estimates, we phase-in these changes by incrementally increasing the sampling weights of the births and decreasing the sampling weights of the inactive EINs in a similar fashion. In the first month, we tabulate the births at one-third their sampling weight and tabulate the inactive EINs at two-thirds their sampling weight. In the second month, we tabulate the births at two-thirds their sampling weight and tabulate the inactive EINs at one-third their sampling weight. In the third month, we tabulate the births at their full sampling weight and the inactive EINs are dropped (sampling weight equal zero).

EINs that were BMF inactive in a previous reference quarter but are identified as BMF active for the current reference quarter are treated as births for this procedure.

Estimation procedures: Estimates of monthly sales and end-of-month inventories as well as quarterly e-commerce sales are derived from data collected in the MRTS. Each month, firms in the MRTS sample are asked to report their sales, e-commerce sales, and end-of-month inventory data for the month just ending. Monthly totals are computed as the sum of weighted data (reported and imputed) for all selected sampling units that meet the sample canvass and tabulation criteria. The weight for a given sampling unit is the reciprocal of its probability of selection into the MRTS sample.

Monthly total estimates for broad industry groups (e.g., 2-, 3-, and 4-digit NAICS levels) are computed by summing the benchmarked monthly totals for the appropriate detailed industries comprising the broader industry group. Quarterly totals are computed similarly.

The monthly totals are then benchmarked to the latest totals from the Annual Retail Trade Survey (ARTS).  Non-employers are included in the ARTS totals, and therefore, the monthly benchmarked totals account for non-employers.

Period-to-period (e.g., month-to-month) change estimates are computed using the benchmarked monthly totals.

Variances are estimated using the method of random groups.

Seasonal adjustment: Estimates are adjusted for seasonal variation and holiday and trading-day effects where appropriate using the X-11 filter-based adjustment procedure available in the Census Bureau’s X-13ARIMA-SEATS (AutoRegressive Integrated Moving Average – Signal Extraction in ARIMA Time Series) program. Seasonal adjustment models are reviewed on an annual basis.

MARTS

Sampling Frame: The MARTS sampling frame contains the active sampling units selected in MRTS.

Stratification: The MARTS units are stratified by broader industry categories and substratified by annual sales size. There are 36 primary strata defined by industry. Within each industry stratum, we stratify the sampling units into 4, 7, 10, or 13 substrata by a measure of size related to their annual sales. We select sampling units expected to have a large effect on the precision of the estimates “with certainty.” This means they are sure to be selected and will represent only themselves (i.e., have a selection probability of 1 and a sampling weight of 1). To identify the certainty units, we determine a substratum boundary (or cutoff) that divides the certainty units from the non-certainty units. We base these cutoffs on a statistical analysis of data extracted from the Census Bureau’s Business Register. We also use this analysis to determine the number and boundaries of non-certainty substrata for each industry group.

Sample Selection: Sample sizes are calculated to meet reliability constraints on estimated annual sales totals for specified industries. Sample selection is done independently within each size stratum using a systematic probability-proportional-to-size procedure where the size used is the MRTS sampling weight. Sampling weights range from 1 to 1,000.

Every two and one-half to three years, the sample is re-selected.  New businesses are not added during the life of the sample. Therefore, as firms go out of business, refuse to respond, etc., the sample deteriorates and becomes less representative. By re-selecting the sample, it better represents current business conditions and many small and medium-size firms are relieved of the early reporting burden.

Estimation procedures: Advance sales estimates for the most detailed industries are computed using a link-relative estimator. For each detailed industry, we compute a ratio of current-to-previous month weighted sales using data from units for which we have obtained usable responses for both the current and previous month. Imputed data for some influential units may also be used in the ratios. Then, for each detailed industry, the advance total sales estimate for the current month is computed by multiplying this ratio by the preliminary sales estimate for the previous month (derived from the larger MRTS) at the appropriate industry level. Total estimates for broader industries are computed as the sum of the detailed industry estimates.

The preliminary sales estimate used in this computation includes data for non-employers (i.e., businesses without paid employees). Therefore, non-employers are represented in the published MARTS estimates. The link-relative estimate is used because there is no systematic tabulation-unit level imputation or adjustment for non-respondents in MARTS. 

Variances are estimated using the method of random groups and are used to determine if measured changes are statistically significant.

Estimates are indirectly benchmarked to the Annual Retail Trade Survey (ARTS) estimates via the link-relative estimation method. 

Seasonal Adjustment: Estimates are adjusted for seasonal variation and holiday and trading-day effects where appropriate using the X-11 filter-based adjustment procedure available in the Census Bureau’s X-13ARIMA-SEATS (AutoRegressive Integrated Moving Average – Signal Extraction in ARIMA Time Series) program. Seasonal adjustment models are reviewed on an annual basis.

2. Procedures for Collecting Information

The sample is a probability sample selected from retail and food services employers (NAICS sectors 44, 45 and 722), contained in the Census Bureau's Business Register, which covers all employers who make social security payments for their employees under the Federal Insurance Contributions Act.

MRTS has approximately 12,000 retail firms included in the survey. Of this number, about 2,500 are large firms and were selected with certainty (100 percent chance of selection). The remaining 9,500 respondents are small and medium sized firms and were selected with non-certainty.

MARTS has approximately 4,700 retail firms included in the survey. Of this number, about 1,250 are large firms and were selected with certainty (100 percent chance of selection). The remaining 3,450 respondents are small and medium sized firms and were selected with non-certainty.

Data for the MRS are obtained at the 6-digit NAICS level with all intermediate and summary tables obtained by summation. Tabulations will include estimates on sales, inventories, and inventories/sales ratios for MRTS, whereas MARTS only includes estimates for sales. Data for the MARTS are published for 3-digit NAICS levels and for select 4-digit NAICS levels and total summary levels. Data for MRTS are published for four-digit and selected five-digit NAICS levels and total summary levels.

Statistical analysis of the monthly data will be based on comparison of the monthly data to:

1. annual survey estimates;

2. prior year monthly and annual survey results;

3. the results of the most recent Economic Census; and

4. published trade, business, and media reports.

Comparisons of the monthly estimates to current and prior year annual and monthly estimates are produced by the use of data edits that identify firms exceeding predetermined tolerance cutoffs. The tolerance cutoffs specify acceptable dollar level and percent differences between the annual data, the previous annual estimates, and monthly data. Data analysis also includes research of inventory-to-sales ratios. Also, an analysis is made at the detailed NAICS level to determine if data reported for each subsector appear acceptable.

For the larger MRTS survey, for 2016, the average response (in terms of total quantity response) to this voluntary survey was about 69 percent for sales and 66 percent for inventories. The 2016 average response (in terms of units) to this voluntary survey was about 51 percent.

For the MARTS survey, for 2016, the average response (in terms of dollar volume) to this survey was about 46 percent while the average unit response was about 50 percent.

3. Methods to Maximize Response

Respondents have the option to receive the MRS by mail, facsimile, or both. They also have the option to report via Internet and would receive a reminder. Responses can be provided via mail, facsimile, telephone, or Internet.

The National Processing Center in Jeffersonville, Indiana, performs a fax reminder and/or telephone follow-up for all firms that have not responded by a certain date. The telephone follow-up is also used if firms have not completely filled out the form or have reported questionable data that may be unacceptable for the estimates.  Firms that refuse to respond to the survey are called to convey the importance of their participation.

A fax machine connected to an "800" telephone line permits fax reporting to our collection facility on a 24-hour basis.  The Census Bureau also has an "800" toll-free telephone number in both Washington, D.C. and Jeffersonville, Indiana, to permit respondents to call in data or ask questions without extra expense.  A Frequently Asked Questions section on our website also provides respondents a simple way to obtain answers to their questions. Respondents can also report online through the "Centurion" system, which allows respondents to report 24 hours a day, 7 days a week, at their convenience. 

Nonresponse Bias Studies

As requested, in 2016 and 2017, the Census Bureau’s Office of Statistical Methods and Research for Economic Programs completed a Nonresponse Bias Analysis for MARTS and MRTS respectively. As part of the studies, response rates were analyzed from April 2013 – December 2015 for both surveys. The reason for using data beginning with April 2013 was to only include observations on the current sample design in the analysis to reduce unknown sources of bias. These studies include analysis of Unit Response Rates (URR) for both surveys as well as Total Quantity Response Rates for MRTS and Dollar Volume Response Rates for MARTS.

The MARTS and MRTS response rate analyses found that there are different reporting patterns observed from the certainty units, typically the larger firms, and noncertainty units, typically small and medium size firms, of each survey. URRs for non-certainty companies are significantly lower than URRs for certainty companies. Some of our key additional efforts to continue to improve the participation of all respondents are outlined below.

Additional Efforts to Maximize Response

To help maximize response, we conduct specialized efforts with companies reluctant to participate in the survey. With the introduction of our latest sample, we re-mailed surveys to companies that declined to participate on the prior sample in hopes of obtaining their response on the current sample. This effort is scheduled to take place at the end of 2017 for the upcoming new sample.

Since the MARTS and MRTS surveys are voluntary, retailers have been increasingly reluctant to supply data. Census has employed a number of new strategies to encourage response, as explained below. We have:

• Redesigned the sample of retailers in the MARTS to prioritize the selection of companies that have been good responders, but without biasing the sample. This has increased the unit response rate from an average of 35.4 percent during the final 6 months of the old MARTS sample (May-October 2015 time period) to 49.7 percent from the beginning of the new MARTS sample through 2016 (November 2015 – December 2016).

• Implemented innovative mailing techniques to increase company awareness of the importance of the survey. We have found that by increasing the use of certified mailings to chronic non-respondents (approximately 1,600 companies) in the MRTS, we successfully increased the response rate by 1.5%. This strategy will be expanded with our new sample mail out beginning in December 2017.

• Reached out to chief economists of large companies through collaboration with the National Association of Business Economists (NABE), asking them to convince their companies to participate in this important survey.

• Worked with the Secretary of Commerce to reach out to CEOs of large companies on the importance of their company’s participation in the MRS. This outreach was either through phone calls or letters.

• Initiated efforts to explore some behavioral science approaches that could additionally incentivize respondents, such as affording instant availability of relevant data upon completion of the survey. Participated in focus groups to discuss these ideas.

• Begun looking at methods we plan to employ to increase response rates in the 2017 Economic Census to see if any of these can be ported over to the monthly surveys.

• Beginning in late 2016, we formed a nonresponse group to discuss reluctant large companies.  The goal of this group is to strategize on the best approach to convince the companies of the importance of reporting on economic indicator surveys such as MRS. This effort has already resulted in some initial success. 

4. Testing of Procedures

We continuously edit the reported data and monitor procedures and methods for data collection in an effort to reduce reporting burden and improve data quality. Budget permitting, we plan to conduct future content evaluation surveys to measure non-sampling error and ways to further reduce reporting burden.

5. Contacts for Statistical Aspects and Data Collection

Questions concerning planning and implementation of this survey are under the direction of Rebecca DeNale, Chief of the Retail Indicators Branch, Census Bureau, (301) 763-2713.

Attachments:

1. MARTS - Press Releases Dec 2016

2. Centurion Online Cover Letter

3. MRTS - Copy of Internet reporting instructions

4. Copy of BEA letter of support – MARTS

5. Copy of BEA letter of support – MRTS

6. Initial Mail Letter

7. Screenshots of Centurion Online

8. MRTS Mail Forms

9. MARTS Mail Forms