Questionnaire Proposal Process example

DRAFT

Date: February 20, 2015

To: BRFSS Working Group Members

From: Brenda Hayes, Deputy Branch Chief, Public Health Surveillance Branch, Division of Population Health.

Topic: Core questions, optional modules, and cognitive testing for the 2017 BRFSS questionnaire

The Public Health Surveillance Branch, (PHSB) in the Division of Population Health, invites programs to submit proposals for new or revised questions/modules to be included on the 2017 Behavioral Risk Factor Surveillance System (BRFSS) Questionnaire. There are a number of issues to consider when developing and proposing optional modules for inclusion by states. First, optional modules that address problems/issues that are relevant at the national and state levels are more likely to be selected by states. Second, shorter optional modules are also more likely to be chosen by states. Finally, states are more likely to include optional modules that are developed with input from state programs and State BRFSS Coordinators.

With the continued interest in state data, increasing demand for local level data, and revised length limitations on the BRFSS questionnaire, the PHSB will follow the policy set forth in the long-term plan concerning financial support for questions included on the 2015 BRFSS instrument. Simply stated, as per our usual policy there will be no charge to CDC programs for questions on the core and rotating core, in 2017. Optional modules will have a 10% increase over the 2017 module fee taking into account the increase in cost of administering the survey, and the costs for data processing. The Current Fee Schedule is on page 3 of this document.

Fees paid to PHSB to fund optional modules are used to support our cooperative agreements with States, Territories, and Pacific Islands, funding the BRFSS in the coming calendar year. Therefore, funding for BRFSS questions for the 2017 questionnaire must be issued from FY16 dollars. All program funds allocated to PHSB for placement of questions on the BRFSS go directly into the State awards. Please review the attached “Funding Information Form” for instructions about transferring funds to PHSB to support optional modules and/or cognitive testing of questions.

The 2017 questionnaire characteristics are as follows:

Financial support from CDC programs is not expected for questions included as core or rotating core.

a. A single set of core questions will be administered to all respondents.

b. The rotating core will continue according to the long-term plan, if possible.

Financial support from CDC programs is expected for optional modules. Optional module questions that are not supported by programs will not be made available for use by States, Territories, and Pacific Islands.

The cost for supporting optional modules ranges from $13,210 to $56,696 per question, based on the module fee schedule (please see attached matrix). New module questions will be charged at $13,210 per question and continuing module questions will be charged based on the number of States which used the module in the most recent previous year, the percentage of respondents to the module during that year and data processing fees.

All new or substantially revised questions to be included on the questionnaire must undergo formal testing at a cognitive laboratory approved by PHSB. The Division or Center proposing questions or changes to existing questions will be responsible for costs associated with testing, and these costs are separate from fees associated with questions on the BRFSS instrument. If a program wishes for the branch to arrange cognitive testing for questions, the cost is $2,000 per question. Funding documents for cognitive testing must be received by BRFSS branch on or before 1 November 2015 otherwise proposed new questions cannot be considered for inclusion in the 2017 questionnaire.

Please note that there is a new application for proposing/modifying a question (included herein). Please complete and return to PHSB, as directed below, the Application for Proposing Questions for the 2017 BRFSS Questionnaire along with supporting materials (the proposal) and the Funding Information Form. All proposals and supporting documents should be submitted to Gloria Colclough GJC2@cdc.gov on or before Monday, 11 May 2015. Please note the Funding Information Form should be submitted to Brenda Hayes, Deputy Branch Chief, Population Health Surveillance Branch at BKH4@CDC.GOV no later than 1 November 2015.

Optional Module Fee Schedule

Per Question Charge* for 2017 BRFSS Modules Based on Number of States Using the Module, the Percent of Respondents Interviewed^1, and Data Processing

Percent of Respondents Interviewed	Number of States Using Module
	≤ 15	16-29	≥ 30
≥ 65	30,015	43,356	56,696
33 - 65	23,347	37,016	50,026
< 33	13,310	23,346	36,686

* All costs are in thousands of dollars per question.

¹ Percent of individuals in each state that would be asked questions in the specific module (i.e. Diabetes Module is only asked to those that answer “yes” to having been diagnosed with diabetes; therefore less than 33% of population can be asked the question).

PREPARING SUBMISSIONS FOR THE

2017 BRFSS QUESTIONNAIRE

A Guide for CDC Programs and

Other Agencies

Table of Contents

Procedures for Adoption/Modification of Questions on the BRFSS 5

Instructions for Submitting a Questionnaire Proposal for the 2017 BRFSS Questionnaire 7

Detailed Instructions: 7

Part I: Application for Proposal for the 2017 BRFSS Questionnaire 9

Section A: Program Information 9

Section B: Type of Application 9

Section C: Section of Questionnaire 9

Section D: Source of Questions 9

Section E: Performance 10

Section F: Public Health Importance 12

Section G: Analytic Plan 13

Section H: Proposed Modifications to the BRFSS Questionnaire 13

Part II: Funding Information Form 14

Procedures for Adoption/Modification of Questions on the BRFSS

The steps listed below are required in order to make changes in the BRFSS questionnaire, to adopt modules for use or to fund previously used modules to be adopted for use in a current year. All questions and modules require the vote and approval of state coordinators during the BRFSS annual meeting. Note that for 2017, the margin of the vote has been increased from 70% to 80%. The procedure has been simplified somewhat from the 2016 process in that only substantive (non-editorial) changes to questions will be reviewed by the Advisory Council. Renewal of funding for existing modules or minor question changes will not go through the Advisory Council, but may require cognitive and field testing as well as state coordinator votes. The forms provided in this document are needed to begin the process (step 1).

1. The first step of the approval process to add or modify questions on the BRFSS is initiated by CDC programs, other federal agencies or entities through a Questionnaire Proposal and Funding Information Form. This proposal includes the source of the question(s), performance (if known—e.g., evidence of validity, reliability, cognitive testing), public health importance, and analytic plan. The two-part forms for submission are provided herein.

2. The completed application is then submitted to the BRFSS Technical Assistance (TA) Team. This team is responsible for progressing questions through the questionnaire development process. It will act as a liaison between stakeholders and the BRFSS Advisory Council for process adherence. The TA Team will receive applications and if necessary recommend to the applicant any preliminary changes.

3. The TA team then forwards the proposals that request substantive (non-editorial) modification of questions and/or propose new questions or modules to the BRFSS Working Group for review. The Working Group and other CDC internal reviewers comprise an Advisory Council which may convene or review items via email to determine whether new question applications adhere to BRFSS’s purpose and if necessary makes recommendations for changes to the questions. It also reviews any available cognitive testing results provided by the applicant and address any issues. Finally, the council recommends (or denies) the application’s advancement to the next step in the process through a preliminary vote.

4. Next, the council’s preliminary approved questions are forwarded for cognitive testing. The cognitive testing usually includes a private focus group during a first round and a small sample of telephone respondents in a second round. The purpose of cognitive testing is to identify question delivery issues that may come up during an actual interview. Based upon the results of cognitive testing, changes may be recommended to the applicant. The applicant is then asked to revise and resubmit.

5. Approved questions are sent to states for review. The questions are then presented by the applicant at the BRFSS Annual Conference/Meeting. BRFSS state coordinators or their designees vote to include or exclude questions from the BRFSS questionnaire. A minimum of 80% approval rate must be achieved for inclusion on the questionnaire.

6. Once questions have achieve the require 80% vote of approval from state coordinators, they are field tested by a host state. This field test is conducted on a sample size of 300 telephone respondents in order to identify any remaining issues with the delivery of new questions. The BRFSS TA Team conducts briefings with BRFSS interviewers before and after their shifts inquiring about issues that may have arisen during the interviews. The interviews are also monitored by the BRFSS TA Team for quality assurance. The compiled data is then sent back to the TA Team and analyzed for any issues. Any remaining issues are again communicated to the applicant and necessary revisions are made to the questions. The questions are also sent for Spanish translation.

7. The survey containing the new and existing questions is sent through CDC clearance. Beginning in 2015 the questionnaire will also be submitted annually for OMB review and approval.

8. Upon OMB approval, the final BRFSS questionnaire is sent to the coordinators for implementation.

9. The Survey Operations Team at CDC PHSB then completes the CATI programming.

10. The survey is then administered at the state-level.

Instructions for Submitting a Questionnaire Proposal for the 2017 BRFSS Questionnaire

This package contains two separate parts which that must be completed and returned to PHSB no later than Monday, 15 September 2015:

Part I: Application for Proposal for the 2017 BRFSS Questionnaire

Part II: Funding Information Form

The application is provided to assist programs/agencies in submitting proposed additions or changes to the BRFSS 2017 survey instrument. This application, along with the requested supporting materials, should be returned to Gloria Colclough, no later than Monday, 11 May 2015. The checklist must be completed with all supporting materials attached for proposals to be considered.

Please send an electronic copy of Part I in Microsoft Word format of the application and supporting materials to:

Gloria J. Colclough

State Support Team

Population Health Surveillance Branch (PHSB)

Division of Population Health

Mailstop F-78

Email: gjc2@cdc.gov

770-488-****

Part II, the Funding Information Form, a separate attachment, should be completed and returned no later than Monday, 11 May 2015 to:

Brenda Hayes

Deputy Branch Chief

Population Health Surveillance Branch (PHSB)

Division of Population Health

Mailstop F-78

770-488-****

Email BKH4@cdc.gov

Detailed Instructions:

Complete only Sections A and B of Part I if the program/agency is not requesting to have questions on the 2017 BRFSS Questionnaire.

Complete Sections A, B, and C of the Part I and the Part II Funding Information Form if the program/agency will support one or more existing optional modules, without modifications to the module(s), on the 2017 BRFSS Questionnaire.

Complete Sections A through H of the Application (Appendix 1) if the progam/agency is requesting modification of questions that are already approved and have been included on prior BRFSS Questionnaires. Complete Part II, the Funding Information Form, if this application is for an Optional Module.

Complete Sections A through G of Part I and Part II, the Funding Information Form, if new emerging core questions are proposed.

Complete Sections A through G of Part I and Part II, the Funding Information Form, if one or more new optional module(s) is/are proposed.

Complete Sections A, B, C, D, and G of Part I, and Part II, the Funding Information Form, if a new analytic plan, new risk variables, and/or new definitions of numerators/denominators are proposed for existing core or modular questions.

Behavioral Risk Factor Surveillance System

Part I: Application for Proposal for the 2017 BRFSS Questionnaire

Please complete this application as carefully and thoroughly as possible.

Incomplete proposals will be returned without review.

Section A: Program Information

Program/Agency Name:

Program/Agency Contact Person:

Email:

Telephone:

Section B: Type of Application

Is this a(n): ☐ MODIFICATION ☐ ADDITION ☐ CONTINUATION (no changes)

☐ NEW PLAN FOR EXISTING CORE OR MODULAR QUESTIONS

☐ PROGRAM WILL HAVE NO QUESTIONS FOR 2017

Section C: Section of Questionnaire

This is for a(n): ☐ OPTIONAL MODULE

☐ EMERGING CORE¹

☐ CORE²

If this is an application for Emerging Core or Core, are you interested in including as an Optional Module if question(s) do not pass state coordinator vote as a core item but passes the vote for an optional module?

☐ Yes

☐ No

Section D: Source of Questions

1. What is the source of the question(s)?

☐ We developed the question(s) {skip to Q4}

☐ The question(s) is/are from an existing instrument or adapted from an existing instrument

Section E: Performance

If not developed by your program then answer Q2-3; otherwise, skip to Q4

2. Please provide the name of the original instrument or source for each question. Please provide links to online sources using the questions if applicable:

3. Are the question(s) modified from the original instrument?

☐ Yes

☐ No

4. Have these questions been part of a human subjects review determination? Y/N

Please enter name of state/agency conducting IRB review_____________________

Please enter protocol # ____________________________________________.

5. Have the question(s) undergone validation testing?

☐ Yes

☐ No

☐ Yes – but not completed

If yes, please provide evidence of the extent of validity testing by providing the following information for each study conducted:

Study title:

Brief description of methods:

Results, including relevant statistics (provide link if available online):

Citation (if applicable):

6. Has the reliability of question(s) been tested?

☐ Yes

☐ No

☐ Yes – but not completed

If yes, please provide evidence of the extent of reliability testing by providing the following information for each study conducted:

Study title:

Brief description of methods:

Results, including relevant statistics (provide link if available online):

Citation (if applicable):

7. Have the question(s) undergone cognitive testing?

☐ Yes (skip next question and go to Date of testing Question)

☐ No

If no, does program want PHSB to have testing conducted?

☐ Yes (go to Question 8) —be sure to complete Part II: Funding Form to provide CAN number(s) to cover costs of cognitive testing.

☐ No (go to Question 8)

If yes, please describe the study design and results:

Date of testing:

Study design:

Results:

Please submit any cognitive testing reports to Dr. Carol Pierannunzi (ivk7@cdc.gov) and copy Gloria Colclough (gjc2@cdc.gov).

8. Have the questions already been administered in surveys or research studies?

☐ Yes

☐ No

If yes, please provide citation(s), links to online information and information on populations to which it was administered:

Citation:

Population:

9. Please indicate approximate total time to administer the set of questions, including instructions.
   ☐ <30s
   ☐ 30s-1min
   ☐ 1-2 min
   ☐ >2 min

☐ Unknown

10. Please indicate the average time to administer per question.
    ☐ <5 seconds
    ☐ 5-10 seconds
    ☐ >10-20 seconds

☐ Unknown

Please provide the methods used to obtain timing data:

11. Are the question(s) telephone-survey ready?

☐ Yes

☐ No

Please describe how you determined readiness of the questions.

Section F: Public Health Importance

12. Please provide a rationale for why the question(s) is/are important to health behavior or chronic disease by addressing the following:

Prevalence or disease burden:

Estimated costs to the public and healthcare:

How the topic is related to a state or national initiative (e.g. Healthy People 2020):

13. Besides your program, how will other states, programs or agencies benefit from the inclusion of these question(s) in the BRFSS?

Section G: Analytic Plan

14. Please explain why state-level estimates are desired (e.g., impact for your program/agency, local/state/national policy implications, support to research funding.)

15. Please explain why there is a need to measure the question(s) over time

16. Please describe how calculated variable(s) will be constructed from the question(s)

1. Please describe how the variable(s) will be used in analyses (e.g., outcome, predictor, etc.).

2. Based on your questions of interest and anticipated effect size, please provide an estimate for required sample size and the rationale/calculations used to determine the size.

Estimated Sample Size:

Rationale/calculations:

Section H: Proposed Modifications to the BRFSS Questionnaire

Please attach additional Word document if space below is not sufficient.

Please name the file 2017BRFSSQUESTPROPOSAL_[AGENCYNAME].DOC

1. Current wording of proposed question(s)

2. New wording of proposed question(s)

3. Explanation and rationale for proposed wording change.

Behavioral Risk Factor Surveillance System

Part II: Funding Information Form

Please provide PHSB with contact information for the person in your program who will be responsible for making arrangements to fund your proposed questions.

Program/Agency contact person for funding issues:

Name:_________________________________________________________________

Email:_________________________________________________________________

Telephone:___________________________________________________________

Fax:___________________________________________________________________

Please complete information on CAN numbers below:
CAN(s) for funding cognitive testing:	CAN(s) for funding questions/ modules:

1385 funding document for cognitive testing must be received by the PHSB on or before 2 November 2015. Please submit the funding at the same time as the modules.

1385 funding documents for supporting optional modules must be received by the PHSB on or before Monday, 2 November 2015.

Is this a new module? (Y/N)	Name of Module(s) (Please use consistent naming for the same module from year to year)	Number of questions included in the module(s)

Contact Brenda Hayes for information about cost of supporting optional modules and funding documents.

Brenda Hayes

Deputy Branch Chief

Population Health Surveillance Branch (PHSB)

Division of Population Health

Mailstop F-78

770-488-****

Email BKH4@cdc.gov

1 Please note that the number of emerging core questions is limited to 4. Proposals with more than 4 questions will not be considered for emerging core.

2 Additions to the BRFSS Standard Core Questionnaire are limited and will be prioritized.

Page 18 of 18