Appendix J EPICCS State Sample Selection Memo

APPENDIX J: EPICCS STATE SAMPLE SELECTION MEMO

Date:

April 29, 2015

To:

Chanchalat Chanhatasilpa

From:

Roline Milfort, Adam Chu, Mamadou Diallo

Subject:

Selection of States for the Erroneous Payments in Child Care Centers Study
(EPICCS)

Attachment:

SAS specifications for selecting the State sample

1.

Introduction

The statistical team has completed the selection of States for participation in EPICCS. The 25
selected States (in alphabetical order) are:
1.
2.
3.
4.
5.

Alabama
Arkansas
California
Colorado
Connecticut

6.
7.
8.
9.
10.

Florida
Georgia
Idaho
Illinois
Kansas

11.
12.
13.
14.
15.

Louisiana
Maryland
Minnesota
Missouri
New Jersey

16.
17.
18.
19.
20.

New York
North Carolina
Ohio
Pennsylvania
South Carolina

21.
22.
23.
24.
25.

Tennessee
Texas
Virginia
Washington
Wisconsin

This memo describes the process and result of the first-stage sample selection of the 25 States listed
above. The sample was selected using systematic sampling with probability-proportionate-to-size
(PPS) with the sampling measure of size (MOS) defined by the average daily attendance (ADA) of
children enrolled in the child care centers for each state. The required ADA counts were obtained
from the FNS National Data Bank (NDB) as of April 4, 2014.

2.

Selection of the State Sample

The sample of 25 States was selected from the frame of 49 States and the District of Columbia
shown in Table 1. Some States were included in the sample with certainty because of their large
ADA relative to the other States. An iterative process was used to determine the certainty States.
The iterative process consisted of the following steps:

APPENDIX J: EPICCS STATE SAMPLE SELECTION MEMO

a.

(1)

Compute the initial probability of selecting a State 𝑝𝑠 and treat as certainty any State
25∗𝑀
(1)
with a probability of selection larger than 1, where 𝑝𝑠 = ∑49 𝑀𝑠 , and 𝑀𝑠 is the ADA
𝑠=1

for State 𝑠.

𝑠

b.

Remove the certainty States obtained at the previous step and compute the probability
of selecting a State using the remaining states. Again treat any state with probability
larger than 1 as a certainty.

c.

Cycle through the previous steps until no computed probability of selection is greater
than 1.

It took three cycles to identify the 9 certainty States, which were: California, Florida, Georgia,
Illinois, New York, North Carolina, Ohio, Pennsylvania, and Texas. As per the study plan, all States
with initial probability of selection greater than 2/3 are treated as “certainties”.
The 16 non-certainty States in the sample were randomly selected among the 40 remaining States
using systematic PPS sampling. States with higher probability of selection had a greater chance of
selection. The randomness of State selection guarantees that the sampling theory can be applied and
therefore measures of uncertainty (variance, confidence intervals, etc.) can be computed using the
probability of selection and other information.
Region and ADA were used to order the frame with an alternating ordering for the variable ADA.
Specifically, the variable ADA was sorted descending for the first region (Northeast), then ascending
for the second region (Midwest), then descending for the third region (South), and finally ascending
for the last region (West). The rationale for the alternating ordering is to have successive States
selected within/across regions to be similar in terms of measure of size (ADA). If it later becomes
necessary to pair States for variance estimation this alternating ordering may help to create more
homogenous pairs of States (in terms of ADA).
Table 2 shows the selected States with the corresponding sampling probabilities and weights.
Attachment 1 provides the program specifications for selecting the sample.

Page 2 of 5

APPENDIX J: EPICCS STATE SAMPLE SELECTION MEMO

Table 1

State sampling frame for the EPICCS

State
Connecticut
Maine
Massachusetts
New Hampshire
New Jersey
New York
Pennsylvania
Rhode Island
Vermont
Illinois
Indiana
Iowa
Kansas
Michigan
Minnesota
Missouri
Nebraska
North Dakota
Ohio
South Dakota
Wisconsin
Alabama
Arkansas
Delaware
District of Columbia
Florida
Georgia
Kentucky
Louisiana
Maryland
Mississippi
North Carolina
Oklahoma
South Carolina
Tennessee
Texas
Virginia
West Virginia
Arizona
California
Colorado
Idaho
Montana
Nevada
New Mexico
Oregon
Utah
Washington
Wyoming
Total

Region
1. NORTHEAST
1. NORTHEAST
1. NORTHEAST
1. NORTHEAST
1. NORTHEAST
1. NORTHEAST
1. NORTHEAST
1. NORTHEAST
1. NORTHEAST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
4. WEST
4. WEST
4. WEST
4. WEST
4. WEST
4. WEST
4. WEST
4. WEST
4. WEST
4. WEST
4. WEST

ADA
15,388
4,413
38,175
7,879
61,184
224,922
121,002
6,918
6,367
106,179
47,303
27,779
23,816
57,146
27,915
53,571
26,843
7,976
98,742
9,247
47,986
38,442
70,391
14,347
5,453
231,571
124,185
48,828
53,648
46,973
38,098
108,665
42,727
27,825
57,193
341,283
54,845
19,864
30,029
380,494
24,008
9,003
8,318
12,420
23,137
31,856
18,453
57,083
5,876
2,945,758

Page 3 of 5

Final Probability
0.204
0.058
0.505
0.104
0.810
1.000
1.000
0.092
0.084
1.000
0.626
0.368
0.315
0.756
0.370
0.709
0.355
0.106
1.000
0.122
0.635
0.509
0.932
0.190
0.072
1.000
1.000
0.646
0.710
0.622
0.504
1.000
0.566
0.368
0.757
1.000
0.726
0.263
0.398
1.000
0.318
0.119
0.110
0.164
0.306
0.422
0.244
0.756
0.078

Final Certainty
0
0
0
0
0
1
1
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
1
1
0
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
0
0
0
0
0
0
9

APPENDIX J: EPICCS STATE SAMPLE SELECTION MEMO

Table 2
STATE
New Jersey
Connecticut
Kansas
Minnesota
Wisconsin
Missouri
Arkansas
Tennessee
Virginia
Louisiana
Maryland
Alabama
South Carolina
Idaho
Colorado
Washington
New York
Pennsylvania
Illinois
Ohio
Florida
Georgia
North Carolina
Texas
California

State sample for the EPICCS
REGION
1. NORTHEAST
1. NORTHEAST
2. MIDWEST
2. MIDWEST
2. MIDWEST
2. MIDWEST
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
4. WEST
4. WEST
4. WEST
1. NORTHEAST
1. NORTHEAST
2. MIDWEST
2. MIDWEST
3. SOUTH
3. SOUTH
3. SOUTH
3. SOUTH
4. WEST

ADA
61184
15388
23816
27915
47986
53571
70391
57193
54845
53648
46973
38442
27825
9003
24008
57083
224922
121002
106179
98742
231571
124185
108665
341283
380494

SAMPLINGPROB
0.810
0.204
0.315
0.370
0.635
0.709
0.932
0.757
0.726
0.710
0.622
0.509
0.368
0.119
0.318
0.756
1
1
1
1
1
1
1
1
1

Page 4 of 5

SAMPLINGWEIGHT
1.235
4.909
3.172
2.706
1.574
1.41
1.073
1.321
1.377
1.408
1.608
1.965
2.715
8.391
3.147
1.323
1
1
1
1
1
1
1
1
1

CERTAINTY
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
1
1
1
1
1
1
1

APPENDIX J: EPICCS STATE SAMPLE SELECTION MEMO

Attachment 1
SAS specifications for selecting the State sample
The code to select the State sample was generated using SAS/BASE/STAT software, Version 9.3 of
the SAS System for Windows.
First convert the input Excel file into a SAS file and call it EPICCS_STATES_FRAME. This file has
49 records, one for each of the 48 contiguous States and the District of Columbia, and three
variables, STATE, REGION and ADA. Split the file into two parts, the first part called
EPICCS_STATES_CERT contains the certainty States and the second part called
EPICCS_STATES_NCERT contains the noncertainty States.
Using the file EPICCS_STATES_NCERT created above, run the following SAS procedure.
PROC SURVEYSELECT DATA=EPICCS_STATES_NCERT METHOD=PPS_SYS
SAMPSIZE=16 OUT=EPICCS_SAMPLE_NCERT SEED=521347;
ID STATE REGION;
SIZE ADA;
RUN;
Append the two files EPICCS_STATES_CERT and EPICCS_SAMPLE_NCERT. For the records
from the former file, set the sampling probabilities and weights to 1.

Page 5 of 5