Attachment J ASPE Sample Size Calculations

CPS Test Questions on Health Insurance: Power Calculations
March 27, 2012
Ben Sommers (ASPE) and Colleen Carey (CEA)
Assumptions:
 The number of individuals per household is fixed at 2.7 (the value among respondents in 2011).
 Desired power is 0.8 and alpha is set at 0.1.
 The total CPS sample is 204,000, but we consider both the possibility of comparison to the
remaining sample in March 2013 and the full sample in March 2014 (i.e., the pretest may be
useful both to show how response changes and to set a baseline).
CENSUS CALCULATION:
For binary outcome with a mean of 3.7%, and a difference of 0.45%, Test Sample needs to be
approximately 13,600 (n ~ 5,000 households).
ALTERNATIVES:
However, the power falls quickly with the prevalence of the binary outcome:
For binary outcome with a mean of 15% (e.g. uninsured), and a difference of 0.5%, Census Test Sample
of 13600 has only approximately 47% power. Test Sample needs to be almost 40,000 (n ~15,000
households) to have 80% power. If we are willing to settle for a full percentage point difference, the
proposed sample is adequate.
For binary outcome with a mean of 65% (e.g. ESI), and a difference of 0.5%, Census Test Sample of
13600 has only approximately 32% power. Test Sample X needs to be almost 80,000 (n ~29,000
households, a large proportion of the full sample) to have 80% power. If we’re willing to settle for a full
percentage point difference, Census Sample of 13,600 has 77% power, and a few hundred more
households (about 600) would give us adequate power.

Mean
3.7%
3.7%
15.0%
15.0%
15.0%
15.0%
15.0%
15.0%
65.0%
65.0%
65.0%
65.0%
65.0%
65.0%

True Difference
0.42%
0.42%
0.50%
0.50%
0.50%
0.50%
1.00%
1.00%
0.50%
0.50%
0.50%
0.50%
1.00%
1.00%

Small Sample
13,600
13,600
13,600
13,600
37,300
39,000
8,200
8,200
13,600
13,600
77,500
90,000
15,200
15,200

Large Sample
204,000
190,400
204,000
190,400
204,000
165,000
204,000
195,800
204,000
190,400
204,000
114,000
204,000
188,800

Power
0.8
0.8
0.4752
0.4738
0.8
0.8
0.8
0.8
0.3246
0.3237
0.8
0.8
0.8
0.8

Source: Lenth, R. V. (2006-9). Java Applets for Power and Sample Size [Computer software]. Retrieved
23 March 2011, from http://www.stat.uiowa.edu/~rlenth/Power .