Sup Stmt Part A

Generic ICR for

BRFSS Preliminary Experiment for Sample and Mode

for the National Center for Chronic Disease Prevention

and Health Promotion

Submitted under

CDC/ATSDR Formative Research and Tool Development

(OMB #0920-1154)

Supporting Statement

Part A

September 8, 2017

List of Attachments:

Attachment 1: BRFSS Data Collection Protocols

Attachment 2: Informed consent script for telephone interviews

Attachment 3: Informed consent script for web-based and mailed questionnaires

Attachment 4: Formatted version of the BRFSS Core Questionnaire for mailing

Attachment 5: Example of letter of invitation to participate in web-based BRFSS

SUMMARY

A Preliminary Experiment test to be conducted under the generic clearance for Preliminary Experiments is requested for the Preliminary Experiment for Sample and Mode for the Behavioral Risk Factor Surveillance System in the National Center for Chronic Disease Prevention and Health Promotion. This generic clearance requests clearance to conduct test of the efficiency of modes of data collection from two sampling frames for survey methodologies under consideration for public health surveillance and programmatic efforts overseen by the National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP), Centers for Disease Control and Prevention (CDC). The purpose of the Preliminary Experiment for Sample and Mode will be to assess the cost effectiveness and data quality from sequential modes of data collection based on telephone and address-based sampling methodologies.

Two types of samples will be tested simultaneously in 5 states:

1) Augmented telephone sample: Using a telephone sample with appended information, including address matching, potential respondents will be divided into 4 groups: landline phone numbers with address matches, landline phone numbers without address matching, cell phone numbers with address matching, and cell phone numbers without address matching. For those numbers with address matches, an invitation to a web survey will be sent by mail. If the respondents do not complete the survey, full questionnaires will be mailed. For those respondents who do not complete mailed surveys, phone calls will be made using the BRFSS current Data Collection Protocols (see Attachment 1). For all landline and cell phone numbers without address matches, calls will be made using the same Data Collection Protocols.

2) Address- Based Sample: Using a United States Postal Service (USPS) address sample, phone numbers will be matched to all addresses. It is anticipated that about 40% of all addresses will be matched to a phone number. For locations without phone numbers, three waves of invitations will be sent: 1) an invitation to a web-based questionnaire, 2) a full printed questionnaire, and: 3) a reminder to participate by either completing the previously mailed questionnaire or completing the survey on the web. For those addresses with phone number matches, the steps above will be appended with a phone call using the BRFSS Data Collection Protocols.

A. Justification

1. Circumstances Making the Collection of Information Necessary

The Centers for Disease Control and Prevention (CDC) requests that the Office of Management and Budget (OMB) approve a generic clearance for the Preliminary Experiment for Sample and Mode using sequential data collection methodologies from telephone and address-based samples. Sampling efficiencies and mode of data collection will be tested under this authorization will be used to inform data collection and surveillance efforts of the NCCDPHP as new data collection methodologies are considered. Preliminary testing of new modes of data collection is essential as personal communication changes. These tests will provide information on changes/improvements in response rates when sequential modes are used with two types of samples (telephone numbers and addresses).

The Behavioral Risk Factor Surveillance System (BRFSS) is the nation's premier system of health-related telephone surveys that collect state data about U.S. and territorial residents regarding their health-related risk behaviors, chronic health conditions, and use of preventive services. Established in 1984 with 15 states, BRFSS now collects data in all 50 states as well as the District of Columbia and three U.S. territories. BRFSS completes more than 400,000 adult interviews each year, making it the largest continuously conducted health survey system in the world. The BRFSS has traditionally used a telephone data collection methodology. The BRFSS sample to produce public use data relied exclusively on landline telephone interviews until 2011. Cell phones have been an increasing large proportion of the sample every year since 2011. Currently about 40% of all interviews are conducted on cell phones. States had the option to append the landline telephone samples to include addresses. This information was used to inform the household that their phone number had been selected for the BRFSS and a call might follow. Traditionally the use of the advance letter in landline sample improved response rates by 12-14%.

When cell phones were introduced in the sample, address matching was not available. The recent improvement in matching cell phone numbers to addresses makes the Augmented Telephone Sample more beneficial than it has been in the past. Address matching to landline numbers has been as high as 60% in previous years. Landline address matching is less reliable than in the past, with approximately 40% of all landline numbers matching to addresses. Cell phone matching has improved over time from a low of 5% two years ago. Addresses matched to cell phone samples is anticipated to be approximately 20-25% in 2017.

The BRFSS has experienced a decline in response rates over several years. The use of advance letters on cell phone samples has not been tried as of this time. Moreover, the BRFSS has never offered respondents any method for providing data other than by phone. This Sampling Preliminary Experiment will make use of both telephone and address-based samples, with matching of phones and locations. As personal communication changes, the BRFSS must explore new modes of reaching participants beyond traditional sampling of telephone numbers.

2. Purpose and Use of Information Collection

The BRFSS has traditionally used a sample of telephone numbers. As noted earlier, recent advancements in telephone sampling has improved the match of cell phone numbers to physical addresses. Concurrently there have been improvements in matching phone numbers to addresses in address-based samples. Whether it is more efficient to maintain a telephone sample that has been matched to an address or to use an address-based sample that has been matched to landline and cell phones is an open question. In addition to questions about phone and address samples, there is ongoing research on the use of sequential modes of response (by telephone interview, mailed questionnaire and/or web survey) to collect health information. Current changes in telecommunications (such as call screening, caller ID and call blocking) have made the reliance on telephone interviewing more expensive as a mode for data collection. Some methods which are less expensive (such as web surveys) result in lower response rates, thereby potentially introducing nonresponse bias. Studies have shown that offering respondents different modes in sequence (rather than concurrently) results in higher response rates. Research on the costs and effectiveness of sequential modes using different samples is sparse. The BRFSS is continually striving to provide valid population-level health information using the most cost-effective and least burdensome method. At the core of the BRFSS methodology is the production of the sample and the means to respond offered to respondents.

The Preliminary Experiment for Sample and Mode will provide a first review of comparison of telephone samples to address-based samples for the BRFSS. Information will be used to direct other experiments of a similar sort. If the results indicate that cost savings can be achieved without diminished validity, additional testing might be undertaken in a larger number of states with a larger number of participants. The final outcomes of such a series of tests might be to allow some respondents to provide information in less expensive modes first and then follow-up with non-respondents using other modes. The research might also determine whether the starting point for a multi-mode data collection protocol should begin with a sample of phone numbers or a sample of addresses. The BRFSS has used a series of preliminary tests before (from 2008-2011), when introducing cell phones into the sample.

The purpose of the Preliminary Experiment for Sample and Mode is to explore new modes of data collection using two types of samples (telephone numbers and addresses) for the BRFSS core questionnaire. Both samples (telephone numbers and addresses) that would be used in this experiment are traditional probability samples. This is not an experiment using any nonprobability sampling approach (such as some internet polling methods). Comparisons of the sequence of data collection modes across two types of samples will allow for understanding of the cost effectiveness and data quality of each method. In addition, cost per complete by sample and mode can be compared to current costs per complete using standard BRFSS data collection protocols. Information derived from the BRFSS Sampling Preliminary Experiment will be used for the following purposes:

1. Determination of the response rates achieved by each mode and for each sample.

2. Determination of the cost per complete for each mode and for each sample.

3. Evaluation of differences in responses (including item refusal, demographic comparisons and health outcomes) by mode and sample.

4. Evaluation of the demographic differences of respondents by mode and sample.

5. Comparison of respondent demographics using the traditional landline/cellphone sample with augmented telephone sample and address based sample.

6. Production of methodological paradata on times of day web-based questionnaires are accessed, calling protocols for reminding respondents reached on the phone that other modes have been provided.

7. Evaluation of the use of new samples including the accuracy of address matching for two types of samples.

8. Comparison of state difference in cost effectiveness, response rates, and data quality (such as item nonresponse) by mode and sample.

The following table shell illustrates some comparisons which can be made across the samples and modes.

Outline of statistical comparison across modes and samples
		Telephone sample
		Address Matched Web-based questionnaire	Address Matched Mailed questionnaire	Address Matched Telephone Interview	Not-Address Matched Telephone Interview
Address- Based Sample	Address Matched Web-based questionnaire	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures
	Address-Matched Mailed Questionnaire		Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures
	Address Matched Telephone Interviewer	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures
	Not Address- matched telephone interview	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures
Traditional BRFSS Protocol Data Collection		Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures	Cost per complete Response rate Data quality measures

No data from the Preliminary Experiment for Sample and Mode will be used for population prevalence estimation or rigorous analysis of health data. Data will be analyzed using to compare presentations of mode, sequence of mode and sample only. It should be noted that phone numbers in the telephone sample which are matched to addresses will not overlap locations or phone numbers in the address-based sample. The two samples are wholly independent of each other and are not linked by the matching process.

Overall results will be used to assess methods for improved data quality and efficiency in the data collection process. The practice of testing samples and modes is standard for all large-scale data collection systems such as the BRFSS and other data collection systems housed within the NCCDPHP. This experiment will provide the first information on whether and how samples for the BRFSS may change. It is currently unknown whether respondents will provide information after being directed to the web or allowed to answer by mail. Results from the experiment will direct further experimentation in BRFSS sampling.

3. Use of Improved Information Technology and Burden Reduction

Computer Assisted Telephone Interview (CATI) software, web-based questionnaire software and other software programs will be used to facilitate the interview process and reduce the time burden on respondents during Preliminary Experiment for Sample and Mode testing when appropriate. In many cases

Efforts to Identify Duplication and Use of Similar Information

Work carried out under this clearance will be designed to address the needs of the NCCDPHP and is not duplicative of other evaluation or test known.

4. Impact on Small Businesses or Other Small Entities

This effort will not have any impact on small businesses or other small entities.

5. Consequences of Collecting the Information Less Frequently

The Preliminary Experiment for Sample and Mode is a one-time data collection request.

6. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

No special circumstances apply.

7. Comments in Response to the Federal Register Notice/Outside Consultation

Section 8A:

As required be 5 CFR 1320.8(d), a 60-day Federal Register Notice was published in the Federal Register for CDC/ATSDR Formative Research and Tool Development (OMB #0920-1154)

Section 8B:

No outside contractors will be used in this Preliminary Experiment.

8. Explanation of any Payment/Gift to Respondents

No incentives/payments will be provided to respondents.

9. Privacy Impact Assessment Information

Items of Information to be Collected

In the BRFSS Preliminary Experiment for Sample and Mode, general demographic characteristics of respondents will be collected and associated with paradata on questions posed to participants and question responses. Respondents will not disclose their names or addresses as part of the question process. Address samples and phone numbers will be kept in separate files and will be securely stored. After completion of the Preliminary Experiment, sample files will be destroyed.

How Information Will Be Shared and For What Purpose

BRFSS Preliminary Experiment for Sample and Mode summary reports will be provided to state and territorial health departments who are partners in the BRFSS. Information in summary form may be used for presentations on methodology, but combined datasets from participating states will not be provided. Individual state health departments will retain data from their states with the above mentioned privacy of address and phone number protocols. Results may also be used to prepare and present methodological research papers at professional conferences or for peer reviewed journals. No data from the Preliminary Experiment test will be used to produce prevalence estimates or analyze public health status.

Impact of the Proposed Collection on Respondents’ Privacy

The Preliminary Experiment sample files will include phone numbers and/or addresses. CDC will not retain any individually identifiable information and will not maintain sample files of phone numbers or addresses. Sample files are kept separate from response files and are not connected to responses. They will be stored in secure locations and destroyed at the end of the Preliminary Experiment.

How Individuals Are Informed That Providing Information Is Voluntary Or Mandatory

Individuals participating in the BRFSS Sample Preliminary Experiment be voluntarily recruited and will be reminded that they may refused to answer specific questions. In all Preliminary Experiment tests where a telephone interview is conducted, protocols for voluntary screening will be used that match procedures for the BRFSS. Attachment 2 provides an example of a telephone screener for the BRFSS. Mailed questionnaires and web-based questionnaires will also inform respondents of the voluntary nature of the data collection in the introduction to the survey (see example language in Attachment 3).

Opportunities to Consent

Verbal consent is obtained from participants during the initial contact (see Attachments 1 and 2). In all Preliminary Experiment test groups, introductory scripts, including the voluntary nature of the survey, will precede the survey questions.

How Information Will Be Secured

Access to the data set will be limited to CDC, a state health department and/or its designated contracted vendor conducting the test. Access to data for the Preliminary Experiment for Sample and Mode will be limited to the program staff members who reviewing the administration and the state health departments (and their designated vendors) collecting the data. Security measures include: 1) Physical controls: CDC facilities are secure, ID accessed buildings. Data will not be stored in hard copy formats; and 2) Technical controls: All electronic data are stored on secured servers protected with firewalls and passwords. All employees are trained on data security measures by taking appropriate HHS courses online. All data collection and records management practices and systems adhere to HHS and CDC IT policies and procedures.

Privacy Act Determination

The Privacy Act does not apply. No IIF will be collected, filed or retrieved by the name of the individual or other unique respondent identifier such as social security number.

10. Justification for Sensitive Questions

The BRFSS core survey contains general questions on health status, health risk behaviors and preventive practices. The core survey includes questions that may be considered sensitive: HIV testing and risk behaviors associated with HIV. These questions must remain on the questionnaire to ensure that the Preliminary Experiment is testing the mode and sample, and not the questionnaire.

11. Estimates of Annualized Hour and Cost Burden

Table 12A shows the estimated annualized burden hours of the BRFSS Preliminary Experiment for Sample and Mode. We anticipate 5 states to participate with 300 completes for each group within each sample. There will be a single data collection.

12A. Estimated Annualized Burden (Hours)

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hrs.)
Telephone Sample
General U.S. Adult Population	Address Matched Web-based questionnaire	300 x 5 states =1,500	1	15/60	375
	Address-Matched Mailed Questionnaire	300 x 5 states =1,500	1	15/60	375
	Address Matched Telephone Interviewer	300 x 5 states =1,500	1	15/60	375
	Not Address- matched telephone interview	300 x 5 states =1,500	1	15/60	375
Address-Based Sample
General U.S. Adult Population	Phone Matched Web-based questionnaire	300 x 5 states =1,500	1	15/60	375
	Phone Matched Mailed Questionnaire	300 x 5 states =1,500	1	15/60	375
	Phone Matched Telephone Interviewer	300 x 5 states =1,500	1	15/60	375
	Not Phone matched Web-based interview	300 x 5 states =1,500	1	15/60	375
	Not Phone Matched Mailed Survey	300 x 5 states =1,500	1	15/60	375
Total		13,500			3,375

12B. Costs to respondents.

No costs to Preliminary Experiment test respondents are anticipated, other than their time to participate. Table 12B shows the estimated cost burden for the single administration of the BRFSS Preliminary Experiment for Sample and Mode, based on the respondent’s time to participate. The total cost burden is estimated to be $82,827.

Estimated Annualized Burden Costs

Table 12B Estimated Annual Cost Burden
Respondent Type	Single Administration Burden Hours	Average Hourly Rate*	Total Single Administration Cost Burden
Telephone Sample
Address Matched Web-based questionnaire	375	$24.54	$9,203
Address-Matched Mailed Questionnaire	375	$24.54	$9,203
Address Matched Telephone Interviewer	375	$24.54	$9,203
Not Address- matched telephone interview	375	$24.54	$9,203
Address-Based Sample
Phone Matched Web-based questionnaire	375	$24.54	$9,203
Phone Matched Mailed Questionnaire	375	$24.54	$9,203
Phone Matched Telephone Interviewer	375	$24.54	$9,203
Not Phone matched Web-based interview	375	$24.54	$9,203
Not Phone Matched Mailed Survey	375	$24.54	$9,203
Total	3,375		$82,827

*Based upon the average hourly earnings from the Current Employment Statistics survey conducted by the Bureau of labor Statistics (available at http://data.bls.gov/cgi-bin/surveymost).

12. Estimates of other Total Annual Cost Burden to Respondents or Recordkeepers/Capital Costs

There are no direct costs to respondents other than their time.

13. Annualized Cost to Federal Government

14. Annualized Estimated Cost to the Federal Government

Estimated funds provided to states	$200,000
Estimated CDC BRFSS budget	50,000
Total	$250,000

15. Explanation for Program Changes or Adjustments

N/A.

16. Plans for Tabulation, Publication, and Project Time Schedule

The following timeline approximates the schedule for the BRFSS Preliminary Experiment for Sample and Mode

	Month 1	Month 2	Month 3	Month 4	Month 5	Month 6	Month 7	Month 8	Month 9
Recruit states
Create website
Order Sample
Telephone sample matched web response
Telephone sample matched mail response
Telephone sample matched interview by phone
Telephone sample unmatched telephone interview
ABS matched web responses
ABS matched mail responses
ABS matched telephone responses
ABS unmatched web responses
ABS unmatched mailed responses
Data analyses and reporting

17. Reason(s) Display of OMB Expiration Date is Inappropriate

Not applicable.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the PRA.

12