Download:
pdf |
pdfBehavioral Risk Factor Surveillance System
2015 Summary Data Quality Report
July 29, 2015
BRFSS 2015 Summary Data Quality Report
1 of 27
Table of Contents
Introduction ..................................................................................................................................................3
Interpretation of BRFSS Response Rates ....................................................................................................4
BRFSS 2015 Call Outcome Measures and Response Rate Formulae .........................................................6
Tables of Outcomes and Rates by State .....................................................................................................11
References ..................................................................................................................................................23
BRFSS 2015 Summary Data Quality Report
2 of 27
Introduction
The Behavioral Risk Factor Surveillance System (BRFSS) is a state-based, CDC-assisted healthdata collection project and partnership of state health departments, CDC’s Division of Population
Health, and other CDC programs and offices. It comprises telephone surveys conducted by the
health departments of all 50 states, the District of Columbia, Puerto Rico, and Guam.
This Summary Data Quality Report presents detailed descriptions of the 2015 BRFSS calling
outcomes and call summary information for each of the states and territories that participated in
the 2015 BRFSS. All BRFSS public-use data are collected by landline telephone and cellular
telephone to produce a single data set aggregated from the 2015 BRFSS territorial- and statelevel data sets. The variables and outcomes provided in this document are applicable to a
combined data set of responses from participants using landline telephones and cellular
telephones within each of the states and territories.
The inclusion of data from cellular telephone interviews in the BRFSS public release data set has
been standard protocol since 2011. In many respects, 2011 was a year of change—both in
BRFSS approach and methodology. As the results of cellular telephone interviews were added in
2011, so were new weighting procedures that could accommodate the inclusion of new
weighting variables. Data users should note that new weighting procedures are likely to affect
trend lines when comparing BRFSS data collected before and after 2011. Because of these
changes, users are advised NOT to make direct comparisons with pre-2011 data, and instead,
should begin new trend lines with that year. Details of changes beginning with the 2011 BRFSS
are provided in the Morbidity and Mortality Weekly Report (MMWR), which highlights
weighting and coverage effects on trend lines.1
The measures presented in this document are designed to summarize the quality of the 2015
BRFSS survey data. Response rates, cooperation rates, and refusal rates for BRFSS are
calculated using standards set by the American Association of Public Opinion Research
(AAPOR).2 The BRFSS has calculated 2015 response rates using AAPOR Response Rate #4,
which is in keeping with rates provided by BRFSS before 2011 using rates from the Council of
American Survey Research Organizations (CASRO).3
On the basis of the AAPOR guidelines, response rate calculations include assumptions of
eligibility among potential respondents or households that are not interviewed. Changes in the
geographic distribution of cellular telephone numbers by telephone companies and the portability
of landline telephone numbers are likely to make it more difficult than in the past to ascertain
which telephone numbers are out-of-sample and which telephone numbers represent likely
households. The BRFSS calculates likely households using the proportions of eligible
BRFSS 2015 Summary Data Quality Report
3 of 27
households among all phone numbers where eligibility has been determined. This eligibility
factor appears in calculations of response-, cooperation-, resolution-, and refusal rates.
Interpretation of BRFSS Response Rates
Because this report reflects the initial inclusion of BRFSS cellular telephone interviews,
contextual information on cellular telephone response rates is provided below. Although cellular
telephone response rates are generally lower than landline telephone response rates across most
surveys, the BRFSS has achieved a cellular telephone response rate that compares favorably with
other similar surveys (Table 1).
Table 1.
Examples of Cellular Telephone and Landline Survey Response Rates
Response Rates
Year(s)
Landline
Cell
Phone
2011–2012
17.0%
18.3%
The Commonwealth Fund 2010 Biennial Health
Insurance Surveyb
2012
29.0%
25.0%
National Immunization Survey (NIS)a c
2014
62.6% a
33.5%
Pew Internet and American Life Projectd
2012
30.0%
20.0%
PSRAI Omnibus Surveye
2015
5.0%
4.0%
2012-2013
47.2%
36.3%
2015
48.2%
47.2%
Survey
California Health Interview Survey (CHIS)
National Adult Tobacco Survey (NATS)f
BRFSSg
c
Unlike the BRFSS, the NIS does not include household sampling in the landline portion of the study but interviews the adult
who self-identifies as the most knowledgeable about household immunization information.
ahttp://healthpolicy.ucla.edu/chis/design/Documents/chis2011-2012-method-2_2014-02-21.pdf
bhttp://www.commonwealthfund.org/interactives-and-data/surveys/2011/mar/2010-biennial-health-insurance-survey
chttp://www.cdc.gov/mmwr/preview/mmwrhtml/mm6433a1.htm
dhttp://www.people-press.org/2006/05/15/the-cell-phone-challenge-to-survey-research/
ehttp://www.pewinternet.org/2015/04/01/appendix-a-about-the-december-week-1-and-week-3-omnibus-survey/
fhttp://www.cdc.gov/tobacco/data_statistics/surveys/nats/pdfs/2012-2013-nats-methodology-final.pdf
gBRFSS
response rates are presented here as median rates for all states and territories.
BRFSS 2015 Summary Data Quality Report
4 of 27
Research by the Pew Research Center indicates that response rates for all telephone-based surveys have
declined in recent years.4 Despite lower response rates, this research supports previous findings5 that
weighting to demographic characteristics of respondents ensures accurate estimates for most measures.
The following tables present landline telephone and cellular telephone calling outcomes and rates. The
BRFSS cellular telephone survey was collected in a manner similar to that of the BRFSS landline
telephone survey. One important difference, however, is that interviews conducted by landline
telephones include random selection among adults within households, while cellular telephone
interviews are conducted with adults who are contacted on personal (nonbusiness) cellular telephones.
The report presents data on three general types of measure by state:
1. Call outcome measures, including response rates, which are based on landline telephone
disposition codes.
2. Call outcome measures, including response rates, which are based on cellular telephone
disposition codes.
3. A weighted response rate, based on a combination of the landline telephone response rate with
the cellular telephone response rate proportional to the total sample used to collect the data for a
state.
For clarity, the BRFSS recommends that authors and researchers referencing BRFSS data quality
include the following language, below. Note the places where authors should include information
specific to their projects.
Response rates for BRFSS are calculated using standards set by the American Association of Public Opinion Research
(AAPOR) Response Rate Formula #4 (http://www.aapor.org/AAPOR_Main/media/publications/StandardDefinitions20169theditionfinal.pdf). The response rate is the number of respondents who completed the survey as a
proportion of all eligible and likely-eligible people. The median survey response rate for all states, territories and
Washington, DC, in 2015 was 47.2, and ranged from 33.9 to 61.1.a Response rates for states and territories included in this
analysis had a median of [provide median] and ranged from [provide range],b For detailed information see the BRFSS
Summary Data Quality Report.c
a
Response rates and ranges should reflect the year(s) included in the analyses.
Response rates for states selected for analysis should be included here. This sentence may be omitted if all states are used
in the analysis.
c
See the Summary Data Quality Report for the year(s) included in the analyses.
b
BRFSS 2015 Summary Data Quality Report
5 of 27
BRFSS 2015 Call Outcome Measures and Response Rate Formulae
The calculations of calling-outcome rates are based on final disposition codes that are assigned after all
calling attempts have been exhausted. The BRFSS may make up to 15 attempts to reach a respondent
before assigning a final disposition code. In 2015, the BRFSS used a single set of disposition codes for
both landline and cell phones, adapted from standardized AAPOR disposition codes for telephone
surveys. A few disposition codes apply only to landline telephone or cellular telephone sample numbers.
For example, answering-device messages may confirm household eligibility for landline telephone
numbers but are not used to determine eligibility of cellular telephone numbers. Disposition codes
reflect whether interviewers have completed or partially completed an interview (1000 level codes),
determined that the household was eligible without completing an interview (2000 level codes),
determined that a household or respondent was ineligible (4000 level codes), or was unable to determine
the eligibility of a household or respondent (3000 level codes). The table below illustrates the codes
used by the BRFSS in 2015, and it notes the instances where codes are used only for landline telephone
or cellular telephone sample numbers.
The Disposition Code Table below uses a number of terms to define and categorize outcomes. These
include the following:
•
•
•
•
•
•
Respondent: A person who is contacted by an interviewer and who may be eligible for interview.
Private residence: Persons residing in private residences or college housing are eligible. Persons
living in group homes, military barracks or other living arrangements are not eligible. Persons
living in vacation homes for 30 days or more are eligible. Eligibility is ascertained by asking
each potential respondent whether they live in a private residence. If the respondent is unsure
whether their residence qualifies, additional definitions of residences are provided.
Landline telephone: A telephone that is used within a specific location, including traditional
household telephones, Voice Over Internet Protocol (VOIP), and Internet phones connected to
computers in a household.
Cellular telephone: A mobile device that is not tied to a specific location for use.
Selected respondent: A person who is eligible for interview. For the cellular telephone sample, a
selected respondent is an adult associated with the phone number who lives in a private residence
or college housing within the United States or territories covered by the BRFSS. For the landline
telephone sample, a selected respondent is the person chosen for interview during the household
enumeration section of the screening questions.
Personal cellular telephone: A cellular telephone that is used for personal calls. Cellular
telephones that are used for both personal and business calls may be categorized as personal
telephones and persons contacted on these phones are eligible for interview. Persons using
telephones that are exclusively for business use are not eligible for interview.
BRFSS 2015 Summary Data Quality Report
6 of 27
Table 2.
2015 Landline Telephone and Cellular Telephone BRFSS Disposition Codes
Category
Interviewed
(1000 level codes)
Eligible, Non-Interview
(2000 level codes)
Unknown Eligibility
BRFSS 2015 Summary Data Quality Report
Code
Description
1100
Completed interview
1200
Partially completed interview
2111
Household level refusal (used for landline only)
2112
Selected respondent refusal
2120
Break off/termination within questionnaire
2210
Selected respondent never available
2220
Household (nonbusiness) answering device
(used for landline only)
2320
Selected respondent physically or mentally unable to
complete interview
2330
Language barrier of selected respondent
3100
Unknown if housing unit
3130
No answer
3140
Answering device, unknown whether eligible
3150
Telecommunication barrier (i.e. call blocking)
3200
Household, not known if respondent eligible
3322
Physical or mental impairment (household level)
3330
Language barrier (household level)
3700
On never-call list
7 of 27
Table 2.
2015 Landline Telephone and Cellular Telephone BRFSS Disposition Codes
Category
Not Eligible
Code
Description
4100
Out of sample
4200
Fax/data/modem
4300
Nonworking/disconnected number
4400
Technological barrier
(i.e., fast busy, phone circuit barriers)
4430
Call forwarding/pager
4450
Cellular telephone number
(used for landline telephone only)
4460
Landline telephone number
(used for cellular telephone only)
4500
Non-residence/business
4510
Group home
4700
Household, no eligible respondent
(teen phone/minor child cellular telephone)
4900
Miscellaneous, non-eligible
Factors affecting the distribution of disposition codes by state include differences in telephone systems,
sample designs, surveyed populations, and data collection processes. Table 3 defines the categories of
disposition codes used to calculate outcome and response rates illustrated in Tables 4A through 6.
Table 3.
Categories of 2015 Landline and Cellular Telephone Disposition Codes
Category
Disposition Code
Definitions
Formulae
Abbreviation
Completed
Interviews
1100+1200
COIN
Eligible
1100+1200+2111+2112+2120+2210+2220+2320+2330
ELIG
Contacted Eligible
1100+1200+2111+2112+2120+2210+2320+2330
CONELIG
BRFSS 2015 Summary Data Quality Report
8 of 27
Table 3.
Categories of 2015 Landline and Cellular Telephone Disposition Codes
Category
Disposition Code
Definitions
Formulae
Abbreviation
Terminations and
Refusals
2111+2112+2120
TERE
Ineligible Phone
Numbers
All 4000 level disposition codes
INELIG
Unknown Whether
Eligible
All 3000 level disposition codes
UNKELIG
Eligibility Factor
ELIG/(ELIG + INELIG)
E
The disposition codes are categorized according to the groups illustrated in Table 3 to produce rates of
resolution, cooperation, completion, refusal and response. In accordance with population surveillance
standards, the proportions of people who may have been eligible for interview, but who were not able to
be interviewed, are accounted for in the formulae.
Eligibility Factor
E = ELIG/ (ELIG + INELIG)
The Eligibility Factor is the proportion of eligible phone numbers from among all sample numbers for
which eligibility has been determined. The eligibility factor, therefore, provides a measure of eligibility
that can be applied to sample numbers with unknown eligibility. The purpose of the eligibility factor is
to estimate the proportion of the sample that is likely to be eligible. The eligibility factor is used in the
calculations of refusal and response rates. Separate eligibility factors are calculated for landline
telephones and cellular telephone samples for each state and territory.
Resolution Rate
((ELIG + INELIG) / (ELIG+INELIG+UNKELIG))*100
The Resolution Rate is the percentage of numbers in the total sample for which eligibility has been
determined. The total number of eligible and ineligible sample phone numbers is divided by the total
number of phone numbers in the entire sample. The result is multiplied by 100 to calculate the
percentage of the sample for which eligibility is determined. Separate resolution rates are calculated for
landline telephone and cellular telephone samples for each state and territory.
BRFSS 2015 Summary Data Quality Report
9 of 27
Interview Completion Rate
(COIN / (COIN + TERE)) * 100
The Interview Completion Rate is the rate of completed interviews among all respondents who have
been determined to be eligible and selected for interviewing. The numerator is the number of complete
and partially completed interviews. This number is divided by the number of completed interviews,
partially completed interviews, and all break offs, refusals, and terminations. The result is multiplied by
100 to provide the percentage of completed interviews among eligible respondents who are contacted by
interviewers. Separate interview completion rates are calculated for landline telephone and cellular
telephone samples for each state and territory.
Cooperation Rate
(COIN / CONELIG) *100
The AAPOR Cooperation Rate is the number of complete and partial complete interviews divided by the
number of contacted and eligible respondents. The BRFSS Cooperation Rate follows the guidelines of
AAPOR Cooperation Rate #2. Separate cooperation rates are calculated for landline telephone and
cellular telephone samples for each state and territory.
Refusal Rate
(TERE / (ELIG + (E * UNKELIG))) * 100
The BRFSS Refusal Rate is the proportion of all eligible respondents who refused to complete an
interview or terminated an interview prior to the threshold required to be considered a partial interview.
Refusals and terminations (TERE) are in the numerator, and the denominator includes all eligible
numbers and a proportion of the numbers with unknown eligibility. The proportion of numbers with
unknown eligibility is determined by the eligibility factor (E as described above). The result is then
multiplied by 100 to provide a percentage of refusals among all eligible and likely to be eligible
numbers in the sample. Separate refusal rates are calculated for landline telephone and cellular telephone
samples for each state and territory.
Response Rate
(COIN / ((ELIG + (E * UNKELIG))) * 100
A Response Rate is an outcome rate with the number of complete and partial interviews in the
numerator and an estimate of the number of eligible units in the sample in the denominator. The BRFSS
Response Rate calculation assumes that the unresolved numbers contain the same percentage of eligible
households or eligible personal cell phones as the records whose eligibility or ineligibility are
determined. The BRFSS Response Rate follows the guidelines for AAPOR Response Rate #4. It also is
similar to the BRFSS CASRO Rates reported prior to 2011. Separate eligibility factors are calculated for
landline telephone and cellular telephone samples for each state and territory and a combined Response
BRFSS 2015 Summary Data Quality Report
10 of 27
Rate for landline telephone and cellular telephone also is calculated. The combined landline telephone
and cellular telephone response rate is generated by weighting to the respective size of the two samples.
The total sample equals the landline telephone sample plus cellular telephone sample. The proportion of
each sample is calculated using the total sample as the denominator. The formulae for the proportions of
the sample are found below:
P1 = TOTAL LANDLINE SAMPLE /
(TOTAL LANDLINE SAMPLE + TOTAL CELL PHONE SAMPLE);
P2 = TOTAL CELL PHONE SAMPLE /
(TOTAL LANDLINE SAMPLE + TOTAL CELL PHONE SAMPLE);
The formula for the Combined Landline Telephone and Cellular Telephone Weighted Response Rate,
therefore, is described below:
COMBINED RESPONSE RATE=
(P1 * LANDLINE RESPONSE RATE) + (P2 * CELL PHONE RESPONSE RATE).
Tables of Outcomes and Rates by State
The tables on the following pages illustrate calling outcomes in categories of eligibility, rates of
cooperation, refusal, resolution, and response by landline telephone and cellular telephone samples.
Tables 4A and 4B provide information on the size of the sample and the numbers and
percentages of completed interviews, cooperation rates, terminations and refusals, and contacts
with eligible households by state and territory.
Tables 5A and 5B provide information on the number and percentage of landline telephone and
cellular telephone sample numbers that are eligible, ineligible, and of unknown eligibility.
Table 6 provides response rates for landline telephone samples, cellular telephone samples, and
combined samples.
BRFSS 2015 Summary Data Quality Report
11 of 27
Table 4A. Landline Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN
State
TERE
N
%
AL
3,453
3.6
AK
2,523
AZ
COOP
Total
Sample
%
N
%
%
2,200
2.3
6,754
7.1
51.1
95,436
1.8
1,005
0.7
4,119
2.9
61.3
143,910
5,270
3.7
2,081
1.5
8,457
6.0
62.3
140,550
AR
4,008
4.0
1,698
1.7
6,633
6.7
60.4
99,270
CA
3,558
2.7
2,578
1.9
6,928
5.2
51.4
134,011
CO
7,424
5.3
1,716
1.2
10,658
7.6
69.7
139,619
CT
7,858
3.6
2,866
1.3
13,100
6.1
60.0
216,048
DE
2,191
2.7
490
0.6
3,345
4.2
65.5
80,310
DC
3,409
2.2
1,310
0.8
5,581
3.6
61.1
155,310
FL
6,087
2.8
3,113
1.4
10,793
5.0
56.4
214,890
GA
3,030
2.2
727
0.5
4,751
3.4
63.8
139,410
HI
2,795
2.2
874
0.7
4,675
3.7
59.8
126,100
ID
3,298
3.6
1,393
1.5
5,306
5.8
62.2
91,260
IL
2,856
3.7
1,062
1.4
4,744
6.2
60.2
77,040
IN
3,941
3.9
1,828
1.8
6,667
6.6
59.1
100,584
IA
3,510
5.5
1,183
1.9
5,325
8.4
65.9
63,270
KS
11,356
5.6
3,929
1.9
16,642
8.2
68.2
203,310
KY
5,345
3.2
1,133
0.7
6,827
4.1
78.3
168,480
LA
2,789
2.9
1,503
1.6
4,909
5.1
56.8
96,403
ME
6,397
6.6
1,725
1.8
9,044
9.3
70.7
97,350
MD
11,007
4.7
4,711
2.0
18,458
7.8
59.6
235,666
MA
4,276
2.6
1,970
1.2
6,527
4.0
65.5
163,001
MI
4,215
4.1
1,166
1.1
6,421
6.2
65.6
104,010
MN
8,007
5.3
1,010
0.7
10,887
7.3
73.5
150,090
MS
3,703
4.0
1,636
1.8
5,989
6.5
61.8
91,713
BRFSS 2015 Summary Data Quality Report
N
CONELIG
12 of 27
Table 4A. Landline Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN
TERE
State
N
%
MO
4,216
5.6
MT
4,048
NE
COOP
Total
Sample
%
N
%
%
1,227
1.6
6,392
8.5
66.0
75,350
5.8
967
1.4
5,824
8.3
69.5
69,786
8,750
6.6
3,009
2.3
13,449
10.1
65.1
133,470
NV
1,826
4.4
563
1.4
2,689
6.5
67.9
41,580
NH
4,997
5.5
1,630
1.8
7,592
8.4
65.8
90,690
NJ
7,811
3.5
1,855
0.8
11,969
5.4
65.3
222,150
NM
3,958
4.5
1,690
1.9
6,491
7.5
61.0
87,120
NY
8,433
3.3
4,567
1.8
15,756
6.1
53.5
259,230
NC
2,447
5.5
946
2.1
3,853
8.7
63.5
44,490
ND
2,978
4.6
752
1.1
4,189
6.4
71.1
65,430
OH
8,706
3.8
1,490
0.7
12,228
5.4
71.2
226,350
OK
4,510
5.1
2,175
2.5
7,888
8.9
57.2
88,360
OR
2,525
3.7
1,000
1.4
3,662
5.3
69.0
69,113
PA
2,787
6.1
1,204
2.6
4,535
10.0
61.5
45,506
RI
4,001
6.3
1,459
2.3
6,347
10.0
63.0
63,390
SC
6,075
6.6
1,578
1.7
9,108
9.9
66.7
91,875
SD
4,297
3.3
980
0.8
6,105
4.7
70.4
129,240
TN
4,090
3.9
1,912
1.8
6,680
6.4
61.2
104,744
TX
9,260
3.0
4,493
1.4
16,163
5.2
57.3
312,870
UT
4,174
5.5
780
1.0
5,871
7.7
71.1
75,766
VT
3,205
7.7
689
1.7
4,380
10.6
73.2
41,400
VA
4,954
5.8
767
0.9
6,927
8.1
71.5
85,650
WA
10,162
4.1
4,256
1.7
16,507
6.7
61.6
247,770
WV
2,969
11.1
661
2.5
4,021
15.0
73.8
26,760
WI
3,176
5.1
1,650
2.7
5,400
8.8
58.8
61,710
BRFSS 2015 Summary Data Quality Report
N
CONELIG
13 of 27
Table 4A. Landline Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN
TERE
State
N
%
WY
4,142
3.2
GU
1,259
PR
COOP
Total
Sample
%
N
%
%
1,532
1.2
6,420
5.0
64.5
128,250
6.1
244
1.2
2,290
11.1
55.0
20,690
2,598
4.9
398
0.8
3,957
7.5
65.7
52,918
Minimum
1,259
1.8
244
0.5
2,290
2.9
51.1
20,690
Maximum
11,356
11.1
4,711
2.7
18,458
15.0
78.3
312,870
Mean
4,805
4.5
1,686
1.5
7,552
7.0
64.0
118,655
Median
4,048
4.1
1,490
1.5
6,421
6.6
63.8
97,350
BRFSS 2015 Summary Data Quality Report
N
CONELIG
14 of 27
Table 4B. Cell Phone Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN
State
TERE
N
%
AL
4,512
6.4
AK
1,134
AZ
COOP
Total
Sample
%
N
%
%
1,155
1.6
5,808
8.2
77.7
70,453
5.1
126
0.6
1,317
5.9
86.1
22,350
2,328
7.7
477
1.6
2,986
9.9
78.0
30,300
AR
1,204
7.9
227
1.5
1,530
10.0
78.7
15,300
CA
8,817
7.6
2,946
2.5
12,362
10.7
71.3
115,717
CO
6,161
10.5
683
1.2
6,986
11.9
88.2
58,680
CT
4,477
6.4
1,013
1.4
5,870
8.3
76.3
70,427
DE
2,036
5.0
292
0.7
2,520
6.2
80.8
40,350
DC
502
3.3
106
0.7
658
4.3
76.3
15,360
FL
2,925
7.1
644
1.6
3,619
8.8
80.8
41,308
GA
1,403
4.9
314
1.1
1,840
6.5
76.3
28,410
HI
4,861
9.6
828
1.6
5,881
11.6
82.7
50,759
ID
2,546
13.8
490
2.7
3,149
17.1
80.9
18,450
IL
2,314
7.4
323
1.0
2,705
8.6
85.5
31,290
IN
2,068
7.9
461
1.8
2,627
10.0
78.7
26,249
IA
2,639
13.2
218
1.1
2,935
14.7
89.9
20,010
KS
12,628
6.8
2,028
1.1
14,871
8.0
84.9
186,569
KY
3,590
4.9
405
0.6
4,067
5.6
88.3
72,930
LA
1,905
5.8
416
1.3
2,370
7.3
80.4
32,582
ME
2,761
10.2
461
1.7
3,321
12.3
83.1
27,060
MD
1,474
7.2
330
1.6
1,928
9.5
76.5
20,400
MA
5,085
3.9
1,933
1.5
7,147
5.5
71.1
129,879
MI
4,896
8.3
1,056
1.8
6,727
11.4
72.8
58,770
MN
8,362
8.3
550
0.5
9,495
9.4
88.1
100,740
MS
2,315
9.4
387
1.6
2,762
11.2
83.8
24,652
MO
2,856
9.4
281
0.9
3,242
10.6
88.1
30,478
BRFSS 2015 Summary Data Quality Report
N
CONELIG
15 of 27
Table 4B. Cell Phone Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN
TERE
State
N
%
MT
2,060
9.3
NE
9,315
NV
%
N
136
0.6
10.8
1,089
988
8.9
NH
2,172
NJ
COOP
Total
Sample
%
%
2,251
10.1
91.5
22,226
1.3
10,890
12.7
85.5
85,890
114
1.0
1,119
10.1
88.3
11,070
7.9
512
1.9
2,860
10.4
75.9
27,510
3,781
5.8
551
0.8
4,779
7.4
79.1
64,860
NM
2,902
10.2
602
2.1
3,570
12.6
81.3
28,380
NY
3,856
6.1
1,107
1.7
5,331
8.4
72.3
63,330
NC
4,063
9.5
595
1.4
4,796
11.2
84.7
42,660
ND
2,212
7.5
330
1.1
2,648
9.0
83.5
29,460
OH
3,116
6.0
288
0.6
3,627
7.0
85.9
51,810
OK
2,312
7.1
1,079
3.3
3,627
11.1
63.7
32,646
OR
2,690
5.5
319
0.6
3,098
6.3
86.8
49,111
PA
2,798
8.9
366
1.2
3,257
10.3
85.9
31,482
RI
2,360
7.3
508
1.6
3,126
9.6
75.5
32,550
SC
5,639
10.7
697
1.3
6,493
12.4
86.8
52,541
SD
3,036
7.3
215
0.5
3,285
7.9
92.4
41,467
TN
1,746
6.8
356
1.4
2,138
8.4
81.7
25,558
TX
4,727
6.6
1,768
2.5
6,784
9.5
69.7
71,700
UT
7,553
16.0
504
1.1
8,516
18.1
88.7
47,100
VT
3,224
8.7
485
1.3
3,886
10.5
83.0
37,170
VA
3,575
8.1
405
0.9
4,280
9.7
83.5
44,250
WA
5,931
7.3
1,925
2.4
8,550
10.6
69.4
80,700
WV
3,177
10.7
281
0.9
3,512
11.8
90.5
29,728
WI
3,130
10.6
769
2.6
4,031
13.7
77.6
29,520
WY
1,378
5.9
194
0.8
1,638
7.0
84.1
23,400
GU
414
6.3
86
1.3
553
8.4
74.9
6,570
BRFSS 2015 Summary Data Quality Report
N
CONELIG
16 of 27
Table 4B. Cell Phone Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN
State
PR
N
TERE
%
N
CONELIG
%
N
COOP
%
%
Total
Sample
2,878
15.8
248
1.4
3,306
18.2
87.1
18,194
Minimum
414
3.3
86
0.5
553
4.3
63.7
6,570
Maximum
12,628
16.0
2,946
3.3
14,871
18.2
92.4
186,569
Mean
3,523
8.1
635
1.4
4,350
9.9
81.4
45,625
Median
2,878
7.6
461
1.3
3,321
9.9
82.7
32,582
BRFSS 2015 Summary Data Quality Report
17 of 27
Table 5A. Landline Sample Categories of Eligibility by State (Landline Only)
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
AL
9,918
10.4
76,716
80.4
8,802
9.2
AK
4,545
3.2
129,811
90.2
9,554
6.6
AZ
10,406
7.4
111,768
79.5
18,376
13.1
AR
7,220
7.3
80,374
81.0
11,676
11.8
CA
8,340
6.2
98,613
73.6
27,058
20.2
CO
11,363
8.1
110,724
79.3
17,532
12.6
CT
16,368
7.6
161,939
75.0
37,741
17.5
DE
3,555
4.4
55,474
69.1
21,281
26.5
DC
7,589
4.9
120,975
77.9
26,746
17.2
FL
13,444
6.3
162,293
75.5
39,153
18.2
GA
5,072
3.6
108,302
77.7
26,036
18.7
HI
5,770
4.6
108,195
85.8
12,135
9.6
ID
5,801
6.4
75,414
82.6
10,045
11.0
IL
7,459
9.7
60,214
78.2
9,367
12.2
IN
7,785
7.7
77,303
76.9
15,496
15.4
IA
5,595
8.8
50,592
80.0
7,083
11.2
KS
17,812
8.8
161,353
79.4
24,145
11.9
KY
7,224
4.3
134,259
79.7
26,997
16.0
LA
6,195
6.4
77,489
80.4
12,719
13.2
ME
9,802
10.1
72,418
74.4
15,130
15.5
MD
21,889
9.3
164,056
69.6
49,721
21.1
MA
7,589
4.7
110,435
67.8
44,977
27.6
MI
7,008
6.7
82,311
79.1
14,691
14.1
MN
11,176
7.4
114,287
76.1
24,627
16.4
MS
8,052
8.8
74,513
81.2
9,148
10.0
MO
6,806
9.0
58,033
77.0
10,511
13.9
MT
6,542
9.4
55,945
80.2
7,299
10.5
BRFSS 2015 Summary Data Quality Report
18 of 27
Table 5A. Landline Sample Categories of Eligibility by State (Landline Only)
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
NE
14,778
11.1
105,613
79.1
13,079
9.8
NV
3,215
7.7
30,813
74.1
7,552
18.2
NH
9,620
10.6
65,601
72.3
15,469
17.1
NJ
12,287
5.5
156,120
70.3
53,743
24.2
NM
6,744
7.7
71,057
81.6
9,319
10.7
NY
18,893
7.3
179,534
69.3
60,803
23.5
NC
5,723
12.9
33,214
74.7
5,553
12.5
ND
4,446
6.8
54,995
84.1
5,989
9.2
OH
12,486
5.5
180,897
79.9
32,967
14.6
OK
8,232
9.3
69,618
78.8
10,510
11.9
OR
3,662
5.3
54,642
79.1
10,809
15.6
PA
4,991
11.0
31,355
68.9
9,160
20.1
RI
7,396
11.7
36,714
57.9
19,280
30.4
SC
10,129
11.0
69,666
75.8
12,080
13.1
SD
6,704
5.2
113,557
87.9
8,979
6.9
TN
9,453
9.0
81,219
77.5
14,072
13.4
TX
23,560
7.5
238,323
76.2
50,987
16.3
UT
5,934
7.8
62,700
82.8
7,132
9.4
VT
4,987
12.0
30,375
73.4
6,038
14.6
VA
7,008
8.2
62,171
72.6
16,471
19.2
WA
24,082
9.7
191,211
77.2
32,477
13.1
WV
4,489
16.8
15,984
59.7
6,287
23.5
WI
6,439
10.4
46,987
76.1
8,284
13.4
WY
9,342
7.3
103,627
80.8
15,281
11.9
GU
2,430
11.7
16,225
78.4
2,035
9.8
PR
4,188
7.9
42,948
81.2
5,782
10.9
Minimum
2,430
3.2
15,984
57.9
2,035
6.6
BRFSS 2015 Summary Data Quality Report
19 of 27
Table 5A. Landline Sample Categories of Eligibility by State (Landline Only)
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
Maximum
24,082
16.8
238,323
90.2
60,803
30.4
Mean
8,897
8.1
91,301
76.9
18,456
15.0
Median
7,396
7.7
77,303
77.9
13,079
13.4
BRFSS 2015 Summary Data Quality Report
20 of 27
Table 5B. Cell Phone Sample Categories of Eligibility by State (Cell Phone Only)
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
AL
5,808
8.2
38,289
54.3
26,356
37.4
AK
1,317
5.9
16,761
75.0
4,272
19.1
AZ
2,986
9.9
13,406
44.2
13,908
45.9
AR
1,530
10.0
7,804
51.0
5,966
39.0
CA
12,362
10.7
42,194
36.5
61,161
52.9
CO
6,986
11.9
26,710
45.5
24,984
42.6
CT
5,870
8.3
26,903
38.2
37,654
53.5
DE
2,520
6.2
17,619
43.7
20,211
50.1
DC
658
4.3
8,375
54.5
6,327
41.2
FL
3,619
8.8
15,328
37.1
22,361
54.1
GA
1,840
6.5
13,838
48.7
12,732
44.8
HI
5,881
11.6
17,730
34.9
27,148
53.5
ID
3,149
17.1
7,624
41.3
7,677
41.6
IL
2,705
8.6
14,968
47.8
13,617
43.5
IN
2,627
10.0
11,220
42.7
12,402
47.2
IA
2,935
14.7
9,302
46.5
7,773
38.8
KS
14,871
8.0
108,824
58.3
62,874
33.7
KY
4,067
5.6
38,581
52.9
30,282
41.5
LA
2,370
7.3
16,935
52.0
13,277
40.7
ME
3,321
12.3
11,343
41.9
12,396
45.8
MD
1,928
9.5
8,672
42.5
9,800
48.0
MA
7,147
5.5
63,248
48.7
59,484
45.8
MI
6,727
11.4
31,130
53.0
20,913
35.6
MN
9,495
9.4
49,818
49.5
41,427
41.1
MS
2,762
11.2
12,874
52.2
9,016
36.6
MO
3,242
10.6
14,676
48.2
12,560
41.2
BRFSS 2015 Summary Data Quality Report
21 of 27
Table 5B. Cell Phone Sample Categories of Eligibility by State (Cell Phone Only)
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
MT
2,251
10.1
12,573
56.6
7,402
33.3
NE
10,890
12.7
50,105
58.3
24,895
29.0
NV
1,119
10.1
4,424
40.0
5,527
49.9
NH
2,860
10.4
12,106
44.0
12,544
45.6
NJ
4,779
7.4
29,042
44.8
31,039
47.9
NM
3,570
12.6
14,898
52.5
9,912
34.9
NY
5,331
8.4
26,059
41.1
31,940
50.4
NC
4,796
11.2
19,648
46.1
18,216
42.7
ND
2,648
9.0
16,639
56.5
10,173
34.5
OH
3,627
7.0
25,955
50.1
22,228
42.9
OK
3,627
11.1
19,082
58.5
9,937
30.4
OR
3,098
6.3
18,746
38.2
27,267
55.5
PA
3,257
10.3
12,974
41.2
15,251
48.4
RI
3,126
9.6
13,737
42.2
15,687
48.2
SC
6,493
12.4
23,979
45.6
22,069
42.0
SD
3,285
7.9
24,857
59.9
13,325
32.1
TN
2,138
8.4
11,441
44.8
11,979
46.9
TX
6,784
9.5
35,367
49.3
29,549
41.2
UT
8,516
18.1
21,749
46.2
16,835
35.7
VT
3,886
10.5
16,298
43.8
16,986
45.7
VA
4,280
9.7
20,701
46.8
19,269
43.5
WA
8,550
10.6
31,098
38.5
41,052
50.9
WV
3,512
11.8
12,035
40.5
14,181
47.7
WI
4,031
13.7
14,850
50.3
10,639
36.0
WY
1,638
7.0
15,567
66.5
6,195
26.5
GU
553
8.4
4,330
65.9
1,687
25.7
PR
3,306
18.2
6,000
33.0
8,888
48.9
BRFSS 2015 Summary Data Quality Report
22 of 27
Table 5B. Cell Phone Sample Categories of Eligibility by State (Cell Phone Only)
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
Minimum
553
4.3
4,330
33.0
1,687
19.1
Maximum
14,871
18.2
108,824
75.0
62,874
55.5
Mean
4,350
9.9
21,836
48.0
19,439
42.1
Median
3,321
9.9
16,639
46.5
14,181
42.7
BRFSS 2015 Summary Data Quality Report
23 of 27
Table 6. Response Rates for Landline Telephone, Cellular Telephone, and Combined Samples
State
Landline Response
Rate
Cell Phone
Response Rate
Combined Response
Rate
AL
31.6
48.6
38.8
AK
51.8
69.6
54.2
AZ
44.0
42.2
43.7
AR
49.0
48.0
48.9
CA
34.0
33.6
33.9
CO
57.1
50.6
55.2
CT
39.6
35.5
38.6
DE
45.3
40.3
43.6
DC
37.2
44.9
37.9
FL
37.0
37.1
37.0
GA
48.6
42.1
47.6
HI
43.8
38.4
42.2
ID
50.6
47.2
50.0
IL
33.6
48.3
37.9
IN
42.8
41.5
42.6
IA
55.7
55.0
55.5
KS
56.2
56.3
56.2
KY
62.1
51.6
59.0
LA
39.1
47.6
41.2
ME
55.1
45.1
52.9
MD
39.7
39.7
39.7
MA
40.8
38.6
39.8
MI
51.7
46.9
49.9
MN
59.9
51.9
56.7
MS
41.4
53.2
43.9
MO
53.3
51.8
52.9
MT
55.4
61.0
56.8
BRFSS 2015 Summary Data Quality Report
21 of 27
Table 6. Response Rates for Landline Telephone, Cellular Telephone, and Combined Samples
State
Landline Response
Rate
Cell Phone
Response Rate
Combined Response
Rate
NE
53.4
60.7
56.3
NV
46.5
44.2
46.0
NH
43.1
41.3
42.7
NJ
48.2
41.3
46.6
NM
52.4
52.9
52.5
NY
34.2
35.9
34.5
NC
37.4
48.5
42.9
ND
60.9
54.7
58.9
OH
59.6
49.1
57.6
OK
48.3
44.3
47.2
OR
58.2
38.6
50.0
PA
44.6
44.3
44.5
RI
37.6
39.1
38.1
SC
52.1
50.4
51.5
SD
59.6
62.7
60.4
TN
37.5
43.4
38.6
TX
32.9
41.0
34.4
UT
63.7
57.0
61.1
VT
54.9
45.1
50.2
VA
57.1
47.2
53.7
WA
36.7
34.1
36.0
WV
50.6
47.3
48.9
WI
42.7
49.7
45.0
WY
39.1
61.9
42.6
GU
46.7
55.6
48.9
PR
55.3
44.5
52.5
Minimum
31.6
33.6
33.9
BRFSS 2015 Summary Data Quality Report
22 of 27
Table 6. Response Rates for Landline Telephone, Cellular Telephone, and Combined Samples
State
Landline Response
Rate
Cell Phone
Response Rate
Combined Response
Rate
Maximum
63.7
69.6
61.1
Mean
47.4
47.2
47.1
Median
48.2
47.2
47.2
BRFSS 2015 Summary Data Quality Report
23 of 27
References
1. Pierannunzi C, Town M, Garvin W, Shaw F, Balluz L. Methodologic changes in the Behavioral Risk
Factor Surveillance System in 2011 and potential effects on prevalence estimates.
MMWR.2012;61(22):410-413. http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6122a3.htm .
Accessed September 5, 2015.
2. The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of
Case Codes and Outcome Rates for Surveys website
http://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf
Accessed June 14, 2016.
3. The Council of American Survey Research Organizations. 2013. Code of Standards and Ethics for
Market, Opinion, and Social Research website.
www.casro.org/resource/resmgr/code/september_2013_revised_code.pdf?hhSearchTerms=%22casro+an
d+response+and+rate%22. Accessed September 5, 2015.
4. The Pew Research Center for People and the Press. 2012. Assessing the Representativeness of Public
Opinion Surveys website. http://www.people-press.org/files/legacypdf/Assessing%20the%20Representativeness%20of%20Public%20Opinion%20Surveys.pdf . Accessed
September 5, 2015.
5. Groves, RM. Nonresponse rates and nonresponse bias in household surveys. Public Opinion
Quarterly. 2006;70(5):646-675.
BRFSS 2015 Summary Data Quality Report
23 of 27
File Type | application/pdf |
File Title | Behavioral Risk Factor Surveillance System |
Subject | BRFSS summary data quality report 2015 |
Author | CDC |
File Modified | 2016-09-14 |
File Created | 2016-08-26 |