Response Memo 2

P.O. Box 2393
Princeton, NJ 08543-2393
Telephone (609) 799-3535
Fax (609) 799-0005
www.mathematica-mpr.com

MEMORANDUM

TO:

Michel Smyth

FROM:

Karen Needels, Heinrich Hock and Pat Nemeth

SUBJECT:

Two Responses to OMB Questions about the OMB
Supporting Statement for the Evaluation of the
Unemployment Compensation Provisions of the American
Recovery and Reinvestment Act of 2009

9/10/2012
UCP-089
DATE:

During a September 5, 2012, discussion, OMB staff requested additional information about
four topics related to the OMB Supporting Statement for the Evaluation of the Unemployment
Compensation Provisions of the American Recovery and Reinvestment Act of 2009 (“the UCP
study”). The four topics are as follows:
1. Additional information about the policies for which estimates of quasi-experimental
impacts will be generated.
2. Comparison between the UCP study and the Rothstein (2011) study.
3. Examples of reports including the use of legislators and lobbyists as sources of data.
4. Information about the use in prior studies of a differential in the incentives provided
to respondents based on whether they complete the UCP study’s recipient survey
through the Internet or by telephone.
As was agreed during the telephone call on September 5th, the information that was
requested is being provided as it becomes available. Therefore, the information included in this
memo pertains to the first and second topics only. Information on the other topics will be
provided separately at a later point.
A. ADDITIONAL INFORMATION ABOUT THE POLICIES FOR WHICH
ESTIMATES OF QUASI-EXPERIMENTAL IMPACTS WILL BE GENERATED
As discussed by telephone on September 5, 2012, recent federal legislation related to the
unemployment compensation (UC) system has been wide-sweeping. Among the many UCrelated provisions that will be examined through the UCP evaluation, the quasi-experimental
impact analysis component of the evaluation will focus on two types of provisions that were
targeted toward benefit recipients. Generally speaking, one type offered additional weeks of
unemployment benefits, while the other increased the after-tax value of benefits, either as an
explicit increase in payment amounts or as an elimination of the taxation of a portion of benefits.
In this discussion, additional information is presented about the policies that will be part of the
An Affirmative Action/Equal Opportunity Employer

MEMO TO: Michel Smyth
FROM:
Karen Needels, Heinrich Hock and Pat Nemeth
DATE:
9/10/2012
PAGE:
2
quasi-experimental analysis and how those policies influence the analytical methods to be used
for the UCP study.
1.

The Policies to Be Examined through the Quasi-Experimental Impacts Analysis

The American Recovery and Reinvestment Act of 2009 (ARRA) contained three provisions
that will be examined through the quasi-experimental impact analysis. One, called Federal
Additional Compensation, was an overall increase of $25 per week in weekly benefit amounts
for all UC recipients. The Federal Additional Compensation was in effect for all states for the
same time, from February 2009 to December 2010. A second provision was a reduction by
$2,400 in the amount of UC benefits that were subject to federal income taxation for calendar
year 2009. This also was available for all states for the same time, but for calendar year 2009
only. The third provision arose because DOL provided incentive funds to states to undertake a
variety of modernization amendments to their basic UC laws; our impact analysis will focus on
the provision that increased UC durations for persons enrolled in approved training programs.1
Under this provision, benefits are extended for 26 weeks for UC recipients who have exhausted
their benefits (known as “exhaustees”) and who are enrolled in and making satisfactory progress
in certain training programs, such as state-approved programs and those authorized by the
Workforce Investment Act.
Other provisions to be examined through the quasi-experimental analysis are part of the
Emergency Unemployment Compensation Act of 2008 (EUC08). (Key pieces of legislation
related to EUC08 are shown in Table 1.) As a result of the recent economic downturn, Congress
established the EUC08 program to provide an extension of benefits beyond what UC recipients
are normally entitled to through the regular UC program. Established in June 2008, this program
initially offered up to 13 weeks of extra benefits for benefit recipients through June 2009, in
addition to the other weeks of benefits that recipients are entitled to. However, subsequent pieces
of legislation modified EUC08 as more information about the strength and severity of the
downturn became apparent. Generally speaking, the additional legislation expanded the number
of weeks of benefits and extended the duration of the program. In November 2008, a second tier
of benefits was added for states with high unemployment rates. Following this, the second tier
was expanded slightly and made no longer contingent upon a state’s economic conditions, while

1

These other modernization provisions pertain to the calculation of eligibility for UI based on an unemployed
worker’s recent prior earnings, whether he or she can qualify for benefits by seeking only part-time work, whether
he or she can qualify for benefits after quitting a job due to family-related responsibilities or situations related to
domestic violence, and the amount of benefits paid to UI recipients with dependents. The impacts of these
provisions will not be examined because few of the states selected for the study adopted the provisions during the
time period covered by the study or because it is expected to be infeasible to isolate the effects of adoption of the
provisions for other reasons.

MEMO TO: Michel Smyth
FROM:
Karen Needels, Heinrich Hock and Pat Nemeth
DATE:
9/10/2012
PAGE:
3
TABLE 1. TIMING OF EUC08 CHANGES IN ENTITLEMENTS
Weeks Available Through:b
Public Law
Number
for the
Legislationa

Date Signed into Last Date of EUC08
Law
Claim Start Date

Last Date of Any
EUC08 Payment

First Tier

Second Tier

Third Tier

Fourth Tier

Notes

110-252

6/30/2008

Cannot occur for a
No payment for a
week of
week beginning after
unemployment ending 6/30/2009
after 3/31/2009

13

n.a.

n.a.

n.a.

Effective for weeks beginning
7/6/2008 and ending 7/12/2008
(in most states); claimants must
have exhausted benefit year on
or after 5/1/2007

110-449

11/21/2008

Cannot occur for a
Last week for benefit
week beginning after collection is the week
3/31/2009
including 8/27/2009

20

13, for states with at
least a 6% TUR

n.a.

n.a.

Additional weeks of benefits
cannot be collected for weeks
before date legislation signed
into law (11/21/2008); second
tier added for high
unemployment states

111-5

2/17/2009

Cannot occur for a
No payment for a
week beginning after week beginning after
12/31/2009
5/31/2010

20

13, for states with at
least a 6% TUR

n.a.

n.a.

No change to the first and
second tiers

111-92

11/6/2009

Cannot occur for a
No payments could
week beginning after be made for weeks
12/31/2009
beginning after
5/31/2010

20

14

13, for states with at
least a 6% TUR

6, for states with at Second tier increased, made
least an 8.5% TUR available to all; third and fourth
tiers added

Various

Various

Various

20

14

13, for states with at
least a 6% TUR

6, for states with at Several pieces of legislation
least an 8.5% TUR extended the time period for
which benefits would be
available, but no changes were
made to the tiers

111-312

12/17/2010

Cannot occur for a
No payments could
week beginning after be made for weeks
1/3/2012
beginning after
6/9/2012

20

14

13, for states with at
least a 6% TUR

6, for states with at No change to the tiers; EB
least an 8.5% TUR trigger look-back period
increased from two years to
three years

Various

MEMO TO: Michel Smyth
FROM:
Karen Needels, Heinrich Hock and Pat Nemeth
DATE:
9/10/2012
PAGE:
4
Weeks Available Through:b
Public Law
Number
for the
Legislationa

Date Signed into Last Date of EUC08
Law
Claim Start Date

Last Date of Any
EUC08 Payment

112-78

12/23/2011

Cannot occur for a
No payments could
week beginning after be made for weeks
3/6/2012
beginning after
8/15/2012

112-96

2/22/2012

Cannot occur for a
No payments could
week beginning after be made for weeks
1/2/2013
beginning after
1/2/2013

Note:
a

First Tier
20

Second Tier
14

20 for weeks 14 for all states for
ending 9/2/2012 weeks ending
or earlier
6/1/2012 or earlier
14 for weeks
ending 9/3/2012
through
12/29/2012

14 weeks for states
with at least a 6%
TUR for weeks
ending between
6/2/2012 and
12/30/2012

Third Tier

Fourth Tier

13, for high
6, for states with at
unemployment states least an 8.5% TUR
with at least a 6%
TUR
13, for states with at
least a 6% TUR for
weeks ending
6/1/2012 or earlier

Notes
No change to the tiers; EB
trigger look-back period
increased from two years to
three yearsc

6 in states on EB or EB trigger look-back period to
16 in states not on end on 12/31/2012c
EB, for states with
at least an 8.5%
TUR for weeks
ending 6/1/2012 or
earlier

13, for states with at
least a 7% TUR for
weeks ending
between 6/2/2012 and 6 weeks for states
9/2/2012
with at least a 9%
TUR for weeks
9, for states with at
ending between
least a 7% TUR for
6/2/2012 and
weeks ending
9/2/2012
between 9/3/2012 and
12/30/2012
10 weeks for states
with at least a 9%
TUR for weeks
ending between
9/3/2012 and
12/30/2012

For simplicity, the table shows only the TUR thresholds for states to become entitled to different tiers of benefits. The table does not show thresholds based on the insured
unemployment rate through which states may become eligible for a tier of benefits.

P.L. 110-252 = The Supplemental Appropriations Act, 2008, Title IV—Emergency Unemployment Compensation; P.L. 110-449 = The Unemployment Compensation Extension Act; P.L.
111-5 = The Assistance for Unemployed Workers and Struggling Families Act, of the American Recovery and Reinvestment Act of 2009, Section 2001 of Division B, Title II; P.L. 111-92 =
The Worker, Homeownership, and Business Assistance Act of 2009 (Worker Assistance Act); P.L. 111-118 = Department of Defense Appropriations Act, 2010; P.L. 111-312 = Tax Relief,
Unemployment Insurance Reauthorization, and Job Creation Act of 2010; P.L. 112-78 = Temporary Payroll Tax Cut Continuation Act of 2011; P.L. 112-96 = Middle Class Tax Relief and
Job Creation Act of 2012.

MEMO TO: Michel Smyth
FROM:
Karen Needels, Heinrich Hock and Pat Nemeth
DATE:
9/10/2012
PAGE:
5
b

Technically, the calculation of the EUC08 maximum benefit amount (MBA) is the lesser of (1) a certain percentage of regular compensation and (2) a certain number of weeks times the
regular average weekly benefit amount (WBA). The calculations to determine EUC08 entitlements are conducted before disqualifications, wage reductions, and other penalties are imposed
upon regular benefits. Dependents’ allowances are included in the calculation, but additional compensation (compensation totally financed by a state and payable under a state law by reason
of high unemployment or other special factors) is not. When the table indicates 13 weeks of benefits, the EUC08 MBA is the lesser of 50 percent of regular compensation and 13 weeks times
the average of the regular WBAs. When the table indicates 6, 14, or 20 weeks of benefits, the EUC08 MBA is the lesser of (1) 24, 54, or 80 percent of regular compensation, respectively,
and (2) 6, 14, or 20 weeks times the average of the regular WBAs, respectively. If the claimant’s regular WBA varies over the course of the benefit collection period, it is possible that the
claimant would get fewer than the full potential number of weeks, because the EUC08 WBA is set to the most recent regular WBA (which could be higher or lower than the average of the
regular WBAs).
c
P.L. 111-312 amended how states can compute if they qualify for EB (or must trigger off EB) by allowing them to use a greater number of prior years as their reference point for their
historical experience. This period is called the “look-back period.” Because of the sustained high unemployment rates in recent years, the legislation allowed states to remain on the EB
program for longer than would otherwise be the case. Although this is a significant portion of the legislation, it will not be examined through a quasi-experimental analysis.
EUC08 = Emergency Unemployment Compensation Act of 2008. n.a. = not applicable. TUR = total unemployment rate.

MEMO TO: Michel Smyth
FROM:
Karen Needels, Heinrich Hock and Pat Nemeth
DATE:
9/10/2012
PAGE:
6
third and fourth tiers—which are contingent upon a state’s economic conditions—were added.
As of November 2009, these four tiers together provided up to 53 total weeks of emergency
benefits (up to 20, 14, 13, and 6 weeks, respectively, for each tier). The structure of the four tiers
of EUC08 benefits remained stable from November 2009 through spring 2012. The program
currently is undergoing a phase-down period and is set to expire at the end of 2012 unless
additional legislation extends it.
In addition to EUC08 benefits, the quasi-experimental analysis will examine the impacts of
recent changes to a permanent standby program called Extended Benefits (EB), which has
existed since 1970 to provide extra weeks of benefits automatically during economic downturns.
Although some of the details of the program have changed since its creation, it has most recently
provided either 13 or 20 weeks of extra benefits. As with some of the EUC08 tiers, the EB
program specifies thresholds of the unemployment rate for benefits to be triggered on.
Historically, the EB program has been financed jointly by the federal government and the states
on a 50-50 basis, and legislators in some states were reluctant to incur the costs associated with
the program. A combination of two recent provisions—which have provided temporary federal
financing for 100 percent of the program and encouraged states to use a supplemental method for
triggering onto the program—greatly increased the number of states that triggered onto the
program than would have otherwise.2 This, in turn, increased the total number of weeks of
benefits that UC recipients could be entitled to. In combination with a regular UI entitlement of
up to 26 weeks and a EUC08 entitlement of up to 53 weeks, a maximum EB entitlement of up to
20 weeks has meant that, in principle, UC recipients could collect up to 99 weeks of benefits.
2.

Rationale for Different Estimation Methods for Different Policies

Table 2 shows the specific provisions that will be examined.3 It also summarizes the sources
of variation for each of the UC policies considered in the impact analysis using data from the
2

The unemployment rate that leads to the triggering on or off of EB benefits could be measured by either the
insured unemployment rate or, if states enacted legislation, the total unemployment rate (TUR). The latter provision
encouraged states that did not previously use the TUR trigger to determine EB eligibility to enact temporary laws to
provide EB benefits if they would qualify for EB through the TUR trigger. Therefore, states that did not already
have the optional TUR trigger on their books could expand the ways in which they could trigger and remain on EB.
Regardless of whether the state already had the TUR trigger on its books, adopted it as a result of ARRA, or chose
not to adopt it, 100 percent of EB benefits were paid by the federal government.
3

Table 2 is very similar to Table B.3 in the submission for OMB clearance. For expository reasons, however,
the table reorders the rows in Table B.3. In addition, Table 2 differs from Table B.3 given that Table 2 does not
show analysis plans of the estimation of impacts of EUC08 Tier 1. The previously planned analysis would have
allowed estimation of the effects of this tier through a comparison of the experiences of (1) recipients who could
have made a smooth transition from regular UI benefits to EUC08 Tier 1 to (2) those who collected all of their
regular UI benefits prior to the enactment of the tier and, therefore, who would have become available for the extra

MEMO TO: Michel Smyth
FROM:
Karen Needels, Heinrich Hock and Pat Nemeth
DATE:
9/10/2012
PAGE:
7
unemployment insurance (UI) recipient survey, which will be administered to a nationally
representative random sample of 2,400 UC recipients. Based on the data, it will be possible to
measure important outcomes, such as recipients’ likelihoods of reemployment, benefit
exhaustion, or participation in training, and the number of weeks of UC benefits collected.
TABLE 2. ESTIMATION OF IMPACTS
Unemployment Compensation Policy

Source of Variation

Method Used

Federal Additional Compensation
Tax exemption on first $2,400 of UI benefits

Timing of availability

ITS

Training-specific 26-week extension

Jurisdiction-by-time variation in availability

DD

EUC08 Tier 3

UI jurisdiction benefit formulas; timing of availability

ITS

EUC08 Tier 2 and 4; Extended Benefits

UI jurisdiction benefit formulas; jurisdiction-by-time
variation in availability; IUR/TUR triggers

DD/RD

DD = differences in differences; EUC08 = Emergency Unemployment Compensation Act of 2008; ITS = interrupted time series;
IUR = insured unemployment rate; RD = regression discontinuity; TUR = total unemployment rate; UI = unemployment insurance.

As seen in the table, the specific methods used to estimate the impact of a given policy
change on outcomes will depend on the details of how the policy became available across states
and over time periods. Changes that occurred across the whole nation at the same time must be
analyzed using an interrupted time series (ITS) design. This is the case for the Federal Additional
Compensation and tax exemption on the first $2,400 of UC benefits, which became available at
the same time for all states. The effects of these policies will be measured as the average
deviation of outcomes from state-specific trends at the moment that the policy was enacted.
Hence, estimated impacts will be interpreted very cautiously because changes in the policy
variables might be capturing common unmeasured characteristics of a cohort or unmeasured
economic shocks faced by the cohort between job loss and the time of observation. In contrast,
because the 26-week extension for UC recipients in approved training programs was adopted by
different states at different times, the differences in differences (DD) analysis for this policy will
be able to take advantage of variation across states over time.
There will be some variation in the analysis approaches used for the different tiers of EUC08
and EB. When the legislation creating EUC08 Tier 2 was enacted, the extra weeks of benefits
(continued)
benefits only after a gap in benefit collection. However, because of concerns about access of administrative data
prior to 2008, the data collection plans were modified to include the collection of administrative and survey data
starting with claimants who had benefit-year-begin dates on January 1, 2008. Thus, an examination of the impacts of
Tier 1 benefits became infeasible.

MEMO TO: Michel Smyth
FROM:
Karen Needels, Heinrich Hock and Pat Nemeth
DATE:
9/10/2012
PAGE:
8
from this tier became available only if a state met specific unemployment rate thresholds. Thus,
some states triggered onto this tier at different points in time, whereas other states did not trigger
on at all. This also is the case for Tier 4. And, in a similar way, states adopted the TUR triggers
for EB benefits at different points in time. Therefore, analysis using either differences in
differences (DD) or an regression discontinuity (RD) approach could be feasible of EUC08 Tiers
2 and 4 and EB.4 However, although the legislation that created EUC08 Tier 3 also specified an
unemployment rate threshold—above which states would qualify for Tier 3 benefits and below
which they would not—an analysis using a DD or RD approach is not expected to be feasible.
This is because, in practice, all except three states triggered onto Tier 3 immediately. In essence,
the unemployment rate threshold for triggering onto Tier 3 was set very low given the economic
conditions that were pervasive in the U.S. at that time, such that there is not meaningful variation
across states in when they triggered onto the tier.
B. COMPARISON BETWEEN THE UCP STUDY AND THE ROTHSTEIN (2011)
STUDY
As requested by OMB, additional information is being provided about comparisons between
the UCP study and the 2011 paper by Jesse Rothstein, in which he examines the consequences of
the recent UC benefit extensions through the EUC08 and EB programs on the unemployment
rate.5 Rothstein draws on data from the Current Population Survey (CPS) and applies quasiexperimental methods; similar to those described in Part B of the OMB clearance package for the
UCP study, to estimate impacts of benefit extensions. Across a variety of empirical
specifications, he finds a small, negative, and statistically significant effect of benefit extensions
on the rate of exit from unemployment. Additional investigations in the study suggest that this
effect was driven by both decreases in the rate of exit from the labor force and reductions in the
reemployment rate.
Although Rothstein’s findings using CPS data add to the body of knowledge about potential
effects of UC policy on outcomes for unemployed workers, the UCP evaluation’s recipient
survey will allow DOL to expand on this previous research in two critical ways:

4

If feasible, the RD approach would rely on the continuous nature of the unemployment rate. In contrast,
analysis of the training-related extension would need to use a DD approach because states either did or did not adopt
the extension.
5

The paper can be found at http://www.brookings.edu/about/projects/bpea/editions/~/media/Projects/
BPEA/Fall%202011/2011b_bpea_rothstein.PDF. Henry Farber and Rob Valletta have written a similar paper, which
is available online at http://www.frbsf.org/economics/economists/rvalletta/uiext.pdf. In this response to OMB, the
focus is on the Rothstein study because it is more thorough, relies on a more rigorous empirical strategy, and
concentrates more sharply on the UC extensions of the recent recession.

MEMO TO: Michel Smyth
FROM:
Karen Needels, Heinrich Hock and Pat Nemeth
DATE:
9/10/2012
PAGE:
9
1. The UCP study’s recipient survey will provide longitudinal information about
individuals who collected UC benefits, DOL’s target population for the evaluation,
which will allow us to avoid the potential bias associated with Rothstein’s use of the
CPS data. As explained more fully in Section A.4 of the OMB clearance package,
the CPS data do not allow job losers who enter into the UC system to be clearly
identified and tracked over their claims spell, because the basic monthly CPS do not
contain information about UC benefit receipt. Therefore, longitudinal analysis of job
losers using the CPS must generally include both UC recipients and nonrecipients.
This implies that the data cannot be used to cleanly estimate the effects of policy
changes on UC recipients—an important limitation that Rothstein points out in his
paper. The impacts of interest to DOL would be diluted across a broader group that
includes nonrecipient job losers, and the extent of this dilution cannot be reliably
calculated. The UCP evaluation will avoid this downward bias by fielding the survey
to individuals identified in states’ administrative records as having received UC
benefits.6
2. The survey will collect data on outcomes that are not measured (or are not well
measured) in the CPS. Although the rates of exit out of unemployment examined by
Rothstein (and to be examined in the UCP study) are of substantial policy interest,
DOL seeks to learn about other important outcomes such as participation in job
search and training services, financial hardships, and access to health care. Further,
as Rothstein points out, the retrospective information in the CPS about the duration
of unemployment appears likely to be subject to various recall biases. As discussed
in the September 5th conference call with OMB, the UCP survey instrument was
developed to minimize such recall biases by, for example, anchoring respondents on
specific events and memorable time periods.
Hence, the fielding of the recipient survey will ensure that DOL has more reliable and
complete information about how UC policies affected the economic well-being of the individuals
directly affected by the UC system.
cc: Jonathan Simonetta

6

The survey data will also be more valuable by being linked to administrative data about individuals’ UC
entitlements. It is necessary to assume for CPS data analysis that all job losers are entitled to collect benefits for the
maximum allowable duration in a state. However, in addition to the sharp difference in the availability of benefits to
UC-eligible and non-eligible individuals, the number of weeks of benefits available through the regular UC, EUC08,
and EB programs to eligible individuals within a state can vary, and such differences are related to earnings prior to
a job loss. Failing to account for variation in entitlements represents measurement error in the policy measure of
interest; thus, relying on CPS data may bias estimated impacts of benefit extensions in unknown ways. The UCP
study will overcome this limitation to prior research by taking into account individual differences in entitlements.