2021BRFSS Att 11 Methods for BFRSS Question Adoption and Mod

Attachment 11: Methods for BRFSS Question Adoption and Modification

The Process for BRFSS Question Adoption and Modification

The process by which BRFSS questions are adopted or modified is outlined in Figure 2 below. The timeline for the adoption of questions, from the time that programs propose them, is approximately 18 months. Since 2018, due to the requirement for OMB approval of new questions, the PHSB has prohibited the introduction of new questions during years when only OMB Change Requests are submitted. Since the current OMB review is valid until March 31, 2022, new questions may be proposed for the 2022-2025 questionnaires. All questions which have been previously reviewed and approved by OMB (whether as modules or core sections) may be used on the questionnaire. A list of OMB approved questions/modules/sections can be obtained from PHSB staff if programs need such information to determine their upcoming questionnaire submissions.

Figure 2: The BRFSS Question Proposal Process

All new, modified or underperforming questions must be cognitively tested. The sponsoring program is required to support the cost of cognitive tests. Beginning in 2020, questions will undergo more robust cognitive tests than in the past. Approximately 30-40 tests will be conducted for each question proposed. Programs should refer to the annual form to propose new questions to anticipate costs for these tests. Programs are highly encouraged to harmonize questions with other federal surveys, when appropriate. Programs should especially take note of the format and wording of similar questions on the National Health Interview Survey (NHIS). The PHSB staff will be available to assist program as they develop question for adoption. The PHSB may be used to administer the cognitive tests, or programs may use the cognitive testing facilities at other federal agencies or private vendors. The cognitive testing reports from sources other than the PHSB must be submitted with the question proposals (see Appendix A). A detailed list of the steps in the process for the changes to the BRFSS questionnaire is provided below. All questions and modules require the vote and approval of state coordinators during the BRFSS questionnaire meeting in 2020. Note that for 2021, the margin of the vote is 70% for modules and 80% for core question updates. The CDC submits new OMB packages for the BRFSS every three years. New questions may be proposed only during years when OMB packages are submitted. In other years, editorial changes to questions will be reviewed by the Advisory Council. Renewal of funding for existing modules or minor question changes will not go through the Advisory Council but may require cognitive and field testing as well as state coordinator votes. The forms provided in this document are needed to begin the process (step 1). Note that in 2020, due in part to the change in the NOFO and in part to the interruption in travel and in-office work due to the COVID response the meeting will be held in October, rather than during the usual schedule of April/May. The steps in the process include:

1. CDC and other federal programs submit the Questionnaire Proposal: The first step in the approval process to add or modify questions on the BRFSS is initiated by CDC programs, other federal agencies or entities through a Questionnaire Proposal and Funding Information Form. This proposal includes the source of the question(s), performance (if known—e.g., evidence of validity, reliability, cognitive testing), public health importance, and analytic plan. The two-part forms for submission are provided herein.

2. Review by BRFSS Technical Assistance Team: The completed application is then submitted to the BRFSS Technical Assistance (TA) Team. This team is responsible for progressing questions through the questionnaire development process. It will act as a liaison between stakeholders and the BRFSS Advisory Council for process adherence. The TA Team reviews applications and if necessary, recommend any preliminary changes.

3. Review by BRFSS Working Group: The TA team will then forward the proposals that request substantive (non-editorial) modification of questions and/or propose new questions or modules to the BRFSS Working Group for review. The Working Group and other CDC internal reviewers comprise an Advisory Council, which may convene or review items via email to determine whether new question applications adhere to BRFSS’s purpose and, if necessary, makes recommendations for changes to the questions. It also reviews any available cognitive testing results provided by the applicant and addresses any issues. Finally, the council recommends (or denies) the application’s advancement to the next step in the process through a preliminary vote.

4. Cognitive Testing: Next, the council’s preliminary approved questions are forwarded for cognitive testing. The cognitive testing usually includes a private focus group during a first round and a small sample of telephone respondents in a second round. The purpose of cognitive testing is to identify question delivery issues that may come up during an actual interview. Based upon the results of cognitive testing, changes may be recommended to the applicant. The applicant is then asked to revise and resubmit.

5. BRFSS Coordinator Vote: Approved questions are sent to states for review. The questions for the 2021 questionnaire are then presented by the sponsor at the BRFSS Questionnaire Meeting in 2020. BRFSS state coordinators or their designees vote to include or exclude questions from the BRFSS questionnaire. A minimum approval rate of 70% for module questions and 80% for core questions must be achieved for inclusion on the questionnaire.

6. Field Testing: Once questions have achieved the required vote of approval from state coordinators, they are field tested by a host state. This field test is conducted on a sample size of 300 telephone respondents in order to identify any remaining issues with the delivery of new questions. The questions included in the field test require annual OMB approval. The BRFSS TA Team conducts briefings with BRFSS interviewers before and after their shifts inquiring about issues that may have arisen during the interviews. The interviews are also monitored by the BRFSS TA Team for quality assurance. The compiled data is then sent back to the TA Team and analyzed for any issues. Any remaining issues are again communicated to the applicant and necessary revisions are made to the questions. The questions are also sent for Spanish translation.

7. CDC Clearance: The survey containing the new and existing questions is sent through CDC clearance and requires annual OMB review and approval.

8. OMB Approval: The questionnaire requires annual OMB review and approval.

9. Implementation: Upon OMB approval, the final BRFSS questionnaire is sent to the coordinators for implementation.

10. CATI Programming: The Survey Operations Team at CDC PHSB then completes the CATI programming.

11. State Administration: The survey is then administered at the state-level.