Attachment 4 – PRAMS Questionnaire Development Process
The PRAMS questionnaire is revised every 3 to 5 years. The goals of each revision process are similar. These goals are to maintain or reduce the length of the core survey; to ensure any emerging topics are captured; and to obtain internal and external partner feedback on the survey questions to ensure continued relevance and utility. The steps in the process, including evaluation of the previous phase questions, cognitive testing, and field testing of new questions, and engagement of states and other partners in the revision process are outlined below and remain constant across revisions.
The first step of the PRAMS questionnaire development process is initiated by the CDC PRAMS team. The CDC PRAMS team distributes the PRAMS Question Solicitation Guidelines widely to jurisdiction and federal colleagues and external partners. This guidance to submit new and revised questions includes the source of the question(s), performance (if known—e.g., evidence of validity, reliability, cognitive testing), public health importance, and analytic plan. The question solicitation period is opened for eight weeks in which invited partners can submit their question requests. In the most recent questionnaire revision, PRAMS received 301 unique requests to add new or revised existing PRAMS questions through the question solicitation process.
Each questionnaire revision provides opportunity to improve question quality and evaluate question relevance and utility. To improve question quality the CDC PRAMS team reviewed data from the current questionnaire to assess question performance in the field and identify questions that may have been challenging for respondents. The CDC PRAMS team decided if these questions should be dropped or require revision before including in the next phase of the questionnaire. To evaluate question relevance and utility, we looked for question redundancy by examining indicators that are common to both the PRAMS survey and the birth certificate data and decided whether certain survey questions could be replaced by birth certificate data. This process included examining published PRAMS validation studies, comparing selected indicators that are on the questionnaire and the birth certificate, and consulting with colleagues from the National Center for Health Statistics on our decisions.
A subgroup of the PRAMS team, the PRAMS Questionnaire Work Group, is responsible for advancing question requests through the questionnaire development process. The work group acts as a liaison between external partners and the PRAMS team for process adherence. The Questionnaire Work Group receives applications and, if necessary, recommend to the applicant any preliminary changes. A Division of Reproductive Health (DRH) Revision Work Group, comprised of CDC DRH subject matter experts in maternal and child health, serves as the questionnaire revision steering committee and assists the PRAMS Questionnaire Work Group in making decisions on the Core survey question selection.
The DRH Revision Work Group convenes and evaluates each application proposal using predetermined evaluation criteria that measures whether or not new question applications align with PRAMS data collection objectives. This group recommends (or denies) the application’s advancement to the next step in the process based on application’s adherence to the evaluation criteria. The work group also votes on new questions that were candidates for Core questions and candidates for the Standard questions.
Next, the PRAMS Questionnaire Work Group presents the accepted proposals to the PRAMS jurisdictions for two rounds of voting. With each successive round of voting, PRAMS jurisdictions narrow the list of existing Core questions to retain, new questions to add to the Core, and moving existing standard questions to the Core survey.
Once PRAMS jurisdictions complete their Core questions selection, the DRH Revision Work Group conducts two additional rounds of voting to further refine the Core survey. All accepted question proposals that are not included in the Core survey are made available in standard questions list.
Accepted questions are formatted for mail, web, and telephone survey administration and translated into Spanish.
English and Spanish mail and telephone versions of new and modified Core questions are sent for cognitive testing. The cognitive testing includes in-depth private interviews with 50 women with diverse racial/ethnic background, education level, and income. The purpose of cognitive testing is to identify question delivery issues that may come up during an actual interview.
Results from cognitive testing are used to inform whether changes to the questions are warranted for clarity and comprehension. The questions then undergo field testing with 40 additional participants from diverse race/ethnic background, education level, and income levels. Field testing is completed at health care provider offices where participants are recruited. Field testing serves as a dress rehearsal prior to field implementation to assess general impression of the survey and ensure that participants can follow survey instructions and complete the self-administered survey. Minor edits to the survey instructions, survey layout, or formatting may be required based on field testing findings. After the questions are finalized, all new and modified Spanish version of the questions are sent to CDC Multilingual Services for a final review to ensure that translations and grammar use are correct.
After IRB and OMB approval is received, the set of Core questions are sent to PRAMS jurisdictions, which select their standard questions and develop any jurisdiction-specific questions based on jurisdiction priorities and input from their steering committees that may consist of internal and external partners such as those from academia, non-profits, health care system, medical associations, etc. PRAMS jurisdictions submit their question selection, and the PRAMS Questionnaire Work Group conducts a final review for accuracy and to ensure that each jurisdiction’s survey adheres to the 14-page, 2-column layout limit. The Work Group then formats the mail survey files for booklet printing, creates the back-up, hard copy phone files, and collaborates with a software development contractor to complete PIDS programming (mail data entry, web, and Computer-Assisted Telephone Interviewing or CATI).
The PRAMS Questionnaire Work Group and staff from each jurisdictions conducts at least three rounds of User-Acceptance Testing to ensure that the survey questions are programmed accurately for mail data entry, web mode, and CATI, prior to implementation.
The survey is then administered at the jurisdiction-level.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | CDC Fact Sheet |
| Author | George Khalil |
| File Modified | 0000-00-00 |
| File Created | 2024-11-24 |