PRAMS SS-B_30 Day_11_13_2024_clean

Attachment 1b. Authorizing Legislation-Safe Motherhood and Infant Health Authorization Attachment 1c. Authorizing Legislation-The Prematurity Research Expansion and Education for

Mothers who deliver Infants Early (PREEMIE) Reauthorization Act of 2018

Attachment 2a. PRAMS Phase 8 Web Survey - MD

Attachment 2b. PRAMS Phase 8 Web Survey – PR

Attachment 2c. PRAMS Phase 8 Web Survey – SC

Attachment 2d. PRAMS Phase 8 Web Survey – VA

Attachment 2e. PRAMS Phase 8 Web Survey - WY

Attachment 2f. PRAMS Phase 8 Web Mode Data Collection Metrics

Attachment 3a. 60-day Federal Register Notice

Attachment 3b. Program Response to Comments 60-day Federal Register Notice

Attachment 4. PRAMS Questionnaire Development Process

Attachment 5. PRAMS Response Gifts and Incentivesby Jurisdiction

Attachment 6. Privacy Impact Assessment

Attachment 7. PRAMS Phase 8 and Phase 9 Questionnaire Topic Reference

Attachment 8a. PRAMS Livebirth Phase 8 Core Mail Questionnaire (English)

Attachment 8b. PRAMS Livebirth Phase 8 Core Mail Questionnaire (Spanish)

Attachment 8c. PRAMS Livebirth Phase 8 Core Phone Questionnaire (English)

Attachment 8d. PRAMS Livebirth Phase 8 Core Phone Questionnaire (Spanish)

Attachment 8e. PRAMS Livebirth Phase 9 Core Mail Questionnaire (English)

Attachment 8f. PRAMS Livebirth Phase 9 Core Mail Questionnaire (Spanish)

Attachment 8g. PRAMS Livebirth Phase 9 Core Phone Questionnaire (English)

Attachment 8h. PRAMS Livebirth Phase 9 Core Phone Questionnaire (Spanish)

Attachment 8i. PRAMS Livebirth Phase 9 Core Web Questionnaire (English)

Attachment 8j. PRAMS Livebirth Phase 9 Core Web Questionnaire (Spanish)

Attachment 9a. PRAMS Stillbirth Mail Questionnaire (English)

Attachment 9b. PRAMS Stillbirth Mail Questionnaire (Spanish)

Attachment 9c. PRAMS Stillbirth Phone Questionnaire (English)

Attachment 9d. PRAMS Stillbirth Phone Questionnaire (Spanish)

Attachment 10a. PRAMS Livebirth Phase 8 Standard Mail Modules (English)

Attachment 10b. PRAMS Livebirth Phase 8 Standard Mail Modules (Spanish)

Attachment 10c. PRAMS Livebirth Phase 8 Standard Phone Modules (English)

Attachment 10d. PRAMS Livebirth Phase 8 Standard Phone Modules (Spanish)

Attachment 10e. PRAMS Livebirth Phase 9 Standard Mail Modules (English)

Attachment 10f. PRAMS Livebirth Phase 9 Standard Mail Modules (Spanish)

Attachment 10g. PRAMS Livebirth Phase 9 Standard Phone Modules (English)

Attachment 10h. PRAMS Livebirth Phase 9 Standard Phone Modules (Spanish)

Attachment 10i. PRAMS Livebirth Phase 9 Standard Web Module (English)

Attachment 10j. PRAMS Livebirth Phase 9 Standard Web Module (Spanish)

Attachment 11a. PRAMS Social Determinants of Health Supplement Mail and Phone (English and Spanish)

Attachment 11b. PRAMS Social Determinants of Health Supplement Web (English and Spanish)

Attachment 11c. PRAMS COVID-19 Vaccine Supplement Mail and Phone (English and Spanish)

Attachment 11d. PRAMS COVID-19 Vaccine Supplement Web (English and Spanish)

Attachment 11e. PRAMS COVID-19 Experiences Supplement Mail and Phone (English and Spanish)

Attachment 11f. PRAMS COVID-19 Experiences Supplement Web (English and Spanish)

Attachment 11g. PRAMS Opioid Use Supplement Mail and Phone (English and Spanish)

Attachment 11h. PRAMS Opioid Use Supplement Web (English and Spanish)

Attachment 11i. PRAMS Marijuana Use Supplement Mail and Phone (English and Spanish)

Attachment 11j. PRAMS Marijuana Use Supplement Web (English and Spanish)

Attachment 11k. PRAMS Disabilities Supplement Mail and Phone (English and Spanish)

Attachment 11l. PRAMS Disabilities Supplement Web (English and Spanish)

Attachment 12a. PRAMS Livebirth Mail Introduction and Informed Consent (English)

Attachment 12b. PRAMS Livebirth Mail Introduction and Informed Consent (Spanish)

Attachment 12c. PRAMS Stillbirth Mail Introduction and Informed Consent (English)

Attachment 12d. PRAMS Stillbirth Mail Introduction and Informed Consent (Spanish)

Attachment 12e. PRAMS Livebirth Phone Introduction and Informed Consent (English)

Attachment 12f. PRAMS Livebirth Phone Introduction and Informed Consent (Spanish)

Attachment 12g. PRAMS Stillbirth Phone Introduction and Informed Consent (English)

Attachment 12h. PRAMS Stillbirth Phone Introduction and Informed Consent (Spanish)

Attachment 12i. PRAMS Livebirth Web Introduction and Informed Consent (English)

Attachment 12j. PRAMS Livebirth Web Introduction and Informed Consent (Spanish)

Attachment 13. IRB Approval

Attachment 14. PRAMS Livebirth Respondent Counts by Jurisdiction

Attachment 15a. PRAMS Field Testing Methodology

Attachment 15b. PRAMS Field Testing Questionniare English and Spanish

Attachment 15c. PRAMS Field Testing Questionnaire Informed Consent (English)

Attachment 15d. PRAMS Field Testing Questionnaire Informed Consent (Spanish)

Attachment 16. Summary of PRAMS Core Question Changes from Phase 8 to Phase 9

Attachment 17. PRAMS Phase 9 Cognitive Testing Report

Attachment 18. PRAMS Phase 9 Field Testing Report

Attachment 19. PRAMS Nonresponse Bias Assessment of 2019 Data

Attachment 19a. PRAMS Nonresponse Bias Assessment of 2020 and 2021 Data

Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The Pregnancy Risk Assessment Monitoring System (PRAMS) is a collaborative project of the Centers for Disease Control and Prevention (CDC) and participating U.S. jurisdictions (comprises of states, cities, and U. S. territories). Beginning in May 2021, PRAMS is funded in 50 jurisdictions. The live births in these jurisdictions are 81% of all live births in the United States . Sites that collect data on women with recent livebirths now include 46 states (excluding Idaho, North Carolina, Ohio, and California), New York City, Washington, DC, Puerto Rico, and the Commonwealth of Northern Mariana Islands (henceforth referred to as ‘jurisdictions or sites’)^¹. Thus, the live births in these jurisdictions and sites make up the sampling frame for each jurisdiction or site participating in PRAMS; as discussed below, the sampling strategy may vary by jurisdiction. Utah is the only site currently funded to collect data on women with a recent stillbirth. PRAMS is a coordinated series of self-administered questionnaires and phone interviews that collect information about maternal experiences and behaviors before and during pregnancy and after pregnancy. Respondents are mothers who have recently given birth to a live born or stillborn infant. Information collection is usually conducted continuously to produce annual estimates; however, a point in time data collection is allowable.

PRAMS is administered through cooperative agreements with jurisdiction health departments. A representative sample of respondents is drawn monthly by each jurisdiction. Each site administers a jurisdiction-tailored questionnaire that consists of (1) a set of core questions administered by all jurisdictions and (2) standard module questions selected by the jurisdictions as desired. Jurisdictions may also include supplemental modules to collect information on emerging issues. CDC may offer funding to support supplemental modules developed by CDC. Participation in the supplemental modules is optional and all participating PRAMS jurisdictions are eligible to apply for supplemental funding for data collection, if available. Jurisdictions may also receive funding from their organization or partners to support implementation of a CDC developed supplement, or a jurisdiction-developed supplement. In addition, CDC or individual jurisdiction may implement call back surveys to gather additional information on post-pregnancy experiences and infant and toddler health. For call back surveys, women who respond to the initial PRAMS survey may be re-contacted (opt-out consent process used) at a later time (beyond 6 months post-birth) to collect additional information about post-pregnancy experiences and infant and toddler health.

Field testing, for which approval is included under this PRA request, will be conducted after cognitive testing to assess clarity of instructions, appropriateness of response options, and overall quality of the question in terms of capturing the desired information in new or substantially revised questions. A convenience sample of no more than 50 women, aged 18 years or older with young infants who are one year of age or less will be recruited in clinics or doctor’s offices to complete the testing questions and then respond to a set of follow-up interview questions to assess appropriateness of response options, clarity of survey instructions, and overall quality of the testing questions in terms of capturing the desired information. Demographic information on age, race/ethnicity, and education level will be collected for the purpose of assessing test results by these characteristics. Attachments 15a and 15b, describe the field-testing methodology and follow-up interview questions, respectively. The full report on Phase 9 field testing is described in Attachment 18. Findings from the Phase 9 field testing showed that most participants (7, 70.0%) reported that instructions were clear. A large proportion of participants (9, 90.0%) reported no difficulty in comprehending the questions while all respondents (10, 100.0%) stated that choosing answers was easy for them. Furthermore, bivariate analyses of the data indicates that there is no significant association between respondents’ spoken language, education, race and perceived quality of instructions, comprehension of questions as well as making choices of answers (p > 0.05). Participants’ perceived quality of instructions, difficulty/ease in the comprehension of questions and difficulty/ease in making choices of answers are not significantly influenced by spoken language, education, and race of participants.

Cognitive testing, which will be conducted under an appropriate generic clearance, such as the National Center for Health Statistics, Collaborating Center for Questionnaire Design Evaluation Research (OMB No. 0920-0222, expiration 9/30/2024), will be completed before field testing to assess reading comprehension and detect response errors for new and heavily modified questions. Surveys questions will be refined based on cognitive testing findings prior to conducting field testing of the revised questions. The entity performing cognitive testing will recruit a convenience sample of no more than 50 women aged 18 years or older with young infants who are one year of age or less to complete the questionnaire and participate in the follow-up cognitive interviews. Question probes are often open-ended to elicit rich narratives designed to reveal information about how respondents interpreted the questions, what aspects of their lives were relevant to those interpretations, and the way they then formulated a response to each question. The full report on Phase 9 cognitive testing is described in Attachment 17, but briefly, cognitive testing revealed five overarching themes or patterns of interpretation that affect how respondents interpret and respond to the testing questions. Respondents made mistakes with question timeframes, often mis-interpreting the preconception timeframe of “12 months before you got pregnant with your new baby” as “12 months before your new baby was born.” Whether a respondent had a prior pregnancy and how old their child was in comparison to their newborn baby was an important variable in understanding some respondents’ answers. Prior pregnancy experiences affected not only respondent viewpoints and confidence approaching medical care, but it also impacted the behaviors and experiences captured in the questionnaire. Many respondents struggled to recall certain conversations that they had with their healthcare providers. While respondents confidently remembered and report on how they felt during certain timeframes (as with the respectful care questions) or their own behaviors, such as whether they drank alcohol during any of the three trimesters of pregnancy, respondents tended to use reason (rather than recall) to answer questions that asked them to report on whether a healthcare provider had ‘asked them’ or ‘talked to them’ about something during their healthcare visits. There were some cases where survey mode (mail versus phone) affected how respondents answered and the types of response error that emerged. Respondents listening to response options read-aloud (phone mode) selected the first answers that seem to fit their experiences before hearing the entire list – causing some to change their answers during administration and resulting in response error for others. In other cases, respondents seemed to lose track of the question intent as the list progressed. This was especially prevalent in the Spanish language testing questionnaire where interviewers found the questions were cumbersome to read aloud. Spanish-language participants had unique interpretations to certain questions (e.g., not having enough to each (food security) is interpreted as not having enough to eat the food that they want to eat) with cases of response errors stemming from translation questions.

Respondent Universe

The target population for PRAMS data collection is mothers who have recently given birth to a live or stillborn infant. An eligible mother is defined as a mother who resides in that jurisdiction and gave birth within that jurisdiction. The following are non-eligible mothers: mothers who are not residents of the jurisdiction, mothers who gave birth outside of the jurisdiction, and mothers who gave birth more than six months before the date of sampling.

Proxy interviews are not conducted within PRAMS.

Jurisdiction-tailored samples

An independent sample is drawn monthly by each jurisdiction. The size of each site sample is pre-determined in collaboration with the jurisdiction health department and CDC. Most sites sample by demographic characteristics (such as race/ethnicity or Medicaid insurance status) though some may sample by health outcomes (such as birth weight) or by geographic region/geostrata (such as public health districts, counties or groups of counties). Jurisdictions may also oversample population groups within their sample, oversampling of minority race or ethnic group(s), low birth weight, or rural counties. To ensure an adequate number of responses for analysis purposes, most jurisdictions must conduct at least 400 questionnaires/interviews annually for each analysis stratum, with most jurisdictions choosing between 2 and 6 strata per jurisdiction.

See Attachment 14 for the estimated size of the annual PRAMS livebirth respondents by jurisdiction. The jurisdiction of Utah is the only jurisdiction currently funded for stillbirth surveillance. It is estimated that there will be approximately 160 responses for the stillbirth surveillance in Utah.

Sampling Frame

PRAMS data collection is conducted through self-administered questionnaires (mail and newly introduced web mode) and telephone interviews. A sample record is one birth certificate or fetal death record originating that month. The PRAMS sample is randomly selected from vital records within each jurisdiction. Each site randomly selects a random sample of mothers from within each sampling stratum, at a sampling rate predetermined by the jurisdiction and CDC. Jurisdictions also have the option to change their sampling approach (e.g., change sampling fraction and/or sample stratification variable) at the start of each surveillance year.

Each jurisdiction obtains the list of birth certificates or fetal death records from their Vital Records department on a monthly basis. For the livebirths survey, once the sample has been drawn from the eligible list, each mother is mailed the PRAMS questionnaire up to three times. Each mailing includes a paper questionnaire and a letter that offers the web mode option so mothers could respond to the survey online using an URL link or QRT code or respond via the mail paper survey. If the mother has not returned the completed paper questionnaire nor responded to the survey online after three such mailings, efforts are made to contact the mother, up to 15 call attempts for each viable phone number, and administer the PRAMS questionnaire via telephone. The stillbirth survey follows the same mailing and telephone protocol as the livebirths, but it will not include an option for web survey.

The sampling frame for each jurisdiction includes all birth certificates from eligible mothers for each month but if a birth certificate is not processed in time for sampling or if the mother’s name is missing from the record, the mother will be excluded from the sampling frame. Records with missing addresses or telephone numbers are included in the sampling frame as jurisdictions can look up missing addresses and telephone numbers for sampled mothers. Birth certificate records that were processed too late and were not included in the sampling frame are accounted for during the weighting process. Once data collection is completed for the birth cohort, PRAMS uses each jurisdiction’s annual cumulative birth file and compare against the monthly sampling frames to adjust for noncoverage of eligible participants that were missed in the monthly sampling frames.

In addition, PRAMS may employ unique sampling strategies for special research questions. For example, the Maternal and Child Health Bureau (MCHB) in the Health Resources and Services Administration (HRSA) used the PRAMS platform to help in the data collection efforts for a national evaluation of the transformed Healthy Start program. MCHB/HRSA randomly selected 15 Healthy Start grantee sites to participate for the one-time oversampling (oversampling of Healthy Start participants for 2017- 2018 only). The goal of the evaluation was to determine the effect of the Healthy Start program on changes in participant-level outcomes (e.g., health services utilization, preventive health behaviors, health outcomes). In these jurisdictions, Healthy Start participants were identified from vital records using personal identifiers provided by Healthy Start grantees (e.g., name, date of birth, date of delivery); PRAMS programs then sampled all identified Healthy Start participants for comparison with non-Healthy Start Participants. Another example of a unique sampling strategy is the opioid-call back survey. For the previously implemented opioid call back surveys implemented starting in October 2019, each participating site has designed a sampling plan to oversample women in areas of the jurisdiction based on metrics of burden selected by the jurisdiction (e.g., opioid overdose or neonatal abstinence syndrome rates).

Table B.1-1. Summary of Similarities and Differences in Sampling by Jurisdiction

Similarities	Differences
Stratified random sampling design, with data collection by mail, web, and telephone follow-up	Sampling designs/geostrata (e.g., stratify sampling by race/ethnicity or birth weight) Oversampling of select populations (e.g., low birth weight or minority racial or ethnic groups) Overall jurisdiction sample size

Weighting Process

Each participating jurisdiction draws a stratified systematic sample every month from a frame of eligible birth certificates. With stratified sampling, jurisdictions may select stratification variable(s) of interest to sample and specific subpopulation(s) to oversample so that inferences for maternal behaviors can be estimated with sufficient precision both at the jurisdiction level and within selected strata. Typically, the annual sample is large enough for jurisdiction-wide estimates within 3.5% at 95% confidence. Estimated proportions within strata are slightly less precise (typically, they are estimated within 5% at 95% confidence).

Each jurisdiction has their unique sampling approach, so the data are weighted by jurisdiction. PRAMS uses the annual cumulative birth file from each jurisdiction to weight their data The weighting process described below is completed for each jurisdiction.

Mothers’ responses are linked to extracted birth certificate data items for weighting. The availability of this information for all sampled women, whether they responded or not, is key to deriving nonresponse weights.

For each respondent, the initial sampling weight is the reciprocal of the sampling fraction applied to the stratum. Sampling fractions in PRAMS range from 1 in 1 (e.g., for the very low birth weight strata in small jurisdictions) to about 1 in 211 (e.g., for the normal birth weight, nonminority strata in populous jurisdictions). Corresponding sampling weights, then, would range from 1 to 211.

Nonresponse adjustment factors attempt to compensate for the tendency of women having certain characteristics (e.g., lower education) to respond at lower rates than women without those characteristics. The rationale for applying nonresponse weights is the assumption that non-respondents would have provided similar answers, on average, to respondents’ answers for that stratum and adjustment category. There are select characteristics that are evaluated for non-response adjustment. These include race/ethnicity, age, education, marital status, county of residence, hospital size, timing of prenatal care, infant birth weight, method of payment for delivery, WIC status, and parity. Where multivariate analysis shows that select characteristics affect the propensity to respond in a stratum, the adjustment factor is the ratio of the sample size in that category to the number of respondents in the category. So that cells with few respondents are not distorted by a few women’s answers, small categories are collapsed until each cell contains at least 25 respondents. The magnitude of the adjustment for nonresponse depends on the response rate for a category. If 80% (or 4/5) of the women in a category respond, the nonresponse weight is 1.25 (or 5/4). Categories with lower response rates have higher nonresponse weights. PRAMS recently completed a nonresponse bias assessment of 2019 data (Attachment 19) and 2020-2021 data (Attachment 19a) using select demographic/socio-economic and health behavior indicators, which showed relatively small actual observed bias. The indicator with the highest mean absolute bias was breastfeeding in hospital (range: 1.18 - 1.95 percentage points). Among the 12 indicators examined, nine to ten had mean absolute bias of less than one percentage point each year. Behaviors indicators (e.g., breastfeeding in hospital, smoking) had the highest level of bias while demographic/socio-economic (e.g., Medicaid status, WIC participation) indicators had lowest. Positive behaviors (e.g., breastfeeding in hospital) were over-estimated and risky behaviors (e.g., smoking during pregnancy) were under-estimated. PRAMS will continue to conduct similar assessments with future birth cohorts. The results using 2020 and 2021 data align with 2019 results and provide justification for our request to remove the historical response rate threshold requirement (currently 50%) for data release.

For each jurisdiction, frame omission studies are carried out to look for problems that occur during frame construction. The frame noncoverage weights are derived by comparing frame files for a year of births to the final calendar year birth file that jurisdictions provided to CDC. Omitted records are usually due to late processing and are evenly scattered across the jurisdiction, but sometimes they are clustered by hospitals or counties or even times of the year. The effect of the noncoverage weights is to bring totals estimated from sample data in line with known totals from the birth file. The magnitude of noncoverage is small (typically from 1% to 5%), so the adjustment factor for noncoverage is not much greater than 1.

Multiplying together the sampling, nonresponse, and noncoverage components of the weight yields the analysis weight. Analyzing PRAMS data requires software that takes into account the complex sampling designs that jurisdictions employ. Such software utilizes first-order Taylor series approximations to calculate appropriate standard errors for the estimates it produces.

PRAMS aggregate data are not weighted to provide national estimates. A detailed description of PRAMS methodology, including standardized data collection methodology, questionnaire and weighting process is available on the PRAMS website at Data Methodology | PRAMS | CDC .

2. Procedures for the Collection of Information

Procedures for collection of information follow common protocols for sampling and questionnaire administration to produce a coordinated series of jurisdiction-tailored surveys with a common reference set of questions (divided into core questions and standard module questions). Some flexibility in the design of the sampling stratification and content of each jurisdiction-tailored questionnaire and operations management is allowed within parameters established by the PRAMS cooperative agreement. Participants in the design and implementation of the data collection process include CDC, jurisdiction health departments, and data collection contractors.

Summary of Steps, Roles, and Responsibilities

CDC PRAMS revises the PRAMS questionnaire in phases every 3-5 years. Each questionnaire revision provides CDC, jurisdictions, and other partners the opportunity to assess current and emerging maternal and infant health topics that that PRAMS can collect information on for the next 3-5 years. These priorities drive the question selection process: which existing survey questions to retain, which existing questions to drop, which new questions to include in the next survey. The process starts with solicitation of new topics from jurisdiction and other partners. Representatives from the PRAMS Site Working Group provide input to CDC on newly proposed question topics. Detailed information about the process is provided in Attachment 4 (PRAMS Questionnaire Development Process). All jurisdictions have the opportunity to vote regarding acceptance of newly proposed questions and content of the core questionnaire. All new or heavily modified candidate questions are sent for cognitive testing then field testing, and after which, undergo additional rounds of voting for core selection. Once the core questionnaire is established, all jurisdictions must use the livebirth core questions unchanged for the entire phase of data collection (Attachments 8a-8d for Phase 8; Attachments 8e-8j for Phase 9). Questions that are approved, but not added to the core, become part of the standard modules list (Attachments 10a-10d for Phase 8; Attachments 10e-10j for Phase 9). At the beginning of each Phase, jurisdictions select their standard modules and may also develop jurisdiction-specific questions based on jurisdiction priorities and input from their jurisdiction steering committees that may consist of internal and external partners such as those from academia, non-profits, health care system, and medical associations. Once the questions are finalized, there are no changes for the length of the questionnaire phase (approximately 3-5 years). The administration time is 26 minutes for a jurisdictions PRAMS survey (15 minutes for core questions, 10 minutes for state selected standard questions and 1 minute for consent). The stillbirth questionnaire (Attachments 9a-9d) was developed based on findings from a collaboration between CDC and Emory University to determine appropriate methods to ask women about their experience with stillbirth, including the types of questions to ask women who had a stillbirth and the appropriate language to use when asking these questions. The stillbirth questionnaire is a standalone survey; standard module questions are not added to the stillbirth survey.
Periodically, to respond to emerging issues, CDC or other federal partners develop and fund jurisdictions to add supplemental modules to the survey mid-questionnaire phase as an insert to the end of the current survey. For the remainder of PRAMS Phase 8, supplemental modules include social determinants of health, COVID-19 vaccine, COVID-19 experiences, prescription and illicit opioid use, marijuana use, and disabilities (Attachments 11a-11l). Jurisdictions may also choose on their own to add questions as a supplement to their survey to address jurisdiction-driven priorities. These questions, however, are not programmed in PIDS and jurisdictions are responsible for data collection outside of PIDS.
Periodically, to respond to emerging issues, CDC or other federal partners fund jurisdictions to do call back surveys. Call back surveys may be implemented to gather additional information on post-pregnancy experiences and infant and toddler health. PRAMS implemented the Opioid call back survey in October 2019, which includes topics such as opioid misuse and access to medication assisted therapy, experiences with respectful care, postpartum care, rapid repeat pregnancy, infant feeding practices, infant health and social services such as well child visit attendance, home visitation, developmental delays, and social supports. The median response rate for participating sites was 59% with a range of 46% to 68%.

Jurisdictions may also choose to perform their own call back surveys to address jurisdiction-driven priorities; such surveys are not required to follow CDC’s procedures (or expectations for) question development or sampling or weighting. For example, the Alaska Childhood Understanding Behavior Study (CUBS) is a follow-up survey to Alaska PRAMS which aims to find out more about the health and early childhood experiences of young children in Alaska. Respondents to the Alaska PRAMS survey are re-contacted once their child turns 3 years old. .

Prior to adding new questions or substantively revise existing ones, PRAMS questions undergo cognitive and field testing. Phase 9 questions underwent cognitive testing by the Collaborating Center for Questionnaire Design and Evaluation Research hosted by NCHS (OMB No. 0920-0222, Exp. 09/30/2024); see report Attachment 17. Field testing for Phase 9 questions was completed under the current PRAMS OMB approval (OMB No. 0920-1273, expiration 11/30/2022); see report Attachment 18. We do not anticipate developing new core survey questions or standard modules during the next three years. PRAMS may conduct cognitive and field testing of new or modified questions for supplemental modules and call back surveys to identify issues with and obtain feedback on the quality of the questions. CDC will submit cognitive testing requests under an appropriate generic clearance. For fielding testing, no more than 50 women aged 18 years or older with infants who are about 1 year old or less will be recruited from clinics or doctor’s offices. Women will be screened for study eligibility using the screener questions in Attachments 15b. If the eligible mothers would like to continue, they will be asked to read the informed consent form prior to participation (Attachments 15c-15d). If they verbally consent, they will be offered a self-administered version of the survey that includes the questions to be tested or the survey will be offered in an interview format. Following the survey, women will participate in follow-up interviews to assess survey question quality (Attachment 15b).
CDC produces the mail survey print files, telephone survey hard copy files, and web survey screens to record web responses, and all data processing documentation. Because jurisdictions select standard and supplemental modules, as well as jurisdiction-added questions, the final questionnaire produced for each jurisdiction is unique to that jurisdiction, although all jurisdictions must include questions from the core.
PRAMS grantee staff send out mailings, which includes options to complete and return a printed version of the survey or to access and complete the survey by web, and conduct follow up telephone interviews for non-responders. Some jurisdiction PRAMS projects have contractors who conduct phone and/or mail operations. CDC provides the data collection software called PRAMS Integrated Data Collection System (PIDS). PIDS is used to schedule and track data collection activities; record data on mail, web, and telephone operations; manage call attempts for telephone interviews; and record survey responses and any additional comments provided by mothers. PIDS includes a component for data entry of mail survey data and Computer-Assisted Telephone Interviewing (CATI) for telephone interviews. All jurisdictions must use PIDS for PRAMS operations, including the CATI software. CDC, through a contractor, has developed a web mode for data collection in PIDS that is integrated alongside the mail and telephone features. Web versions of the survey have been programmed in PIDS for the five early adopter sites to deploy in May 2022. Based on examination of key metrics (e.g., response rates, item nonresponse), all PRAMS jurisdictions will implement the web mode during Phase 9.
CDC works with each jurisdiction to determine its sample size and sampling stratification. Jurisdiction PRAMS staff coordinate with their jurisdiction Vital Records Department to create the sampling program and implement it on a monthly basis. Monthly operations span across approximately 90 days, after mothers are sampled. The timeline can be customized by the jurisdiction as long as operations for a batch are completed within the 90-day limit. This time frame allows for approximately 6 weeks of mail and web attempts (up to 3 mailings), and 6 weeks of phone follow-up for non-responders for each monthly batch. As mail surveys are returned, jurisdictions complete data entry for mail responses into PIDS and conduct quality assurance and validation checks before submitting data to CDC.
PRAMS awardees are responsible for field operations and determine how their data will be collected within the PRAMS protocol. Jurisdictions may collect data using in-house staff or hire vendors using Request for Proposal (RFP) procedures, or contract with universities. Data collectors must develop and maintain procedures to ensure respondents’ privacy, assure and document the quality of the interviewing process, and supervise and monitor the interviewers.
Jurisdictions release de-identified data files to CDC on a monthly basis for cleaning and weighting. CDC returns clean, weighted data files to the jurisdiction of origin for its use. Jurisdiction data are released for use by internal and external researchers. In 2024, CDC PRAMS transitioned to an automated process for providing access for a pre-approved list of variables (or data set), in order to disseminate data in a more timely and efficient manner. Researchers may submit a request to access data following instructions found on the PRAMS web site (https://www.cdc.gov/prams/prams-data/researchers.htm).
Maternal and child health indicators derived from PRAMS data are available in aggregate and by jurisdiction: Selected 2016-2022 Maternal and Child Health (MCH) Indicators | PRAMS | CDC . These aggregate estimates are not nationally representative, but instead reflect the combined estimates from jurisdictions included in the dataset and the estimates are proportional to the number of live births in each jurisdiction.

Content and Construction of the PRAMS Questionnaire(s)

The PRAMS questionnaire is comprised of a core which includes questions asked of all respondents, standard module questions which are preselected at the beginning of each phase by the jurisdictions, and jurisdiction-added questions which are developed by jurisdictions. Supplemental modules for emerging issues may be added to the survey mid-questionnaire phase as an insert to the end of the current survey. Call back surveys may be implemented to gather additional information on post-pregnancy experiences and infant and toddler health. Jurisdictions may also choose to perform their own supplements and call back surveys to address jurisdiction-driven priorities.

Attachments 7a and 7b provides a questionnaire topic reference list for Phase 8 and Phase 9 surveys, respectively. All questions included in the PRAMS core survey, standard modules, supplemental modules and call back survey are cognitively tested, and field tested prior to inclusion in the questionnaire.

Core Questions: This portion of the questionnaire must be asked by all sites (Attachment 8a-8d for Phase 8; Attachments 8e-8j for Phase 9). It remains fixed for all jurisdictions across the entire questionnaire phase.
Standard Modules: Sets of standardized questions on various topics that each jurisdiction may select are available as standard modules (Attachment 10a-10d for Phase 8; Attachments 10e-10j for Phase 9).
Jurisdiction-added Questions: Jurisdictions may choose to gather data on additional topics related to their specific health priorities using extra questions they choose to add to their questionnaire. The CDC reviews and makes suggestions on the questions; however, jurisdictions ultimately decide on the final jurisdiction-added questions.
Supplemental Modules: Jurisdictions may be funded to rapidly implement CDC developed supplemental modules developed mid-Phase to address emerging issues (planned supplemental modules for 2022 social determinants of health, COVID-19 vaccine, COVID-19 experiences, opioid use, marijuana use, and disabilities) (Attachments 11a-11l).
Call back Surveys: Jurisdictions may be funded to implement call back surveys to gather additional information on post-pregnancy experiences and infant and toddler health. Women who respond to the PRAMS survey may be re-contacted (opt-out consent process used) at a later time to collect additional information about post-pregnancy experiences and infant and toddler health
Cognitive and Field Testing: New or substantively revised questions would undergo cognitive and field testing prior to implementation. CDC will submit cognitive testing requests under an appropriate generic clearance. For field testing, participants will be recruited at clinics or doctor’s offices from among women aged 18 years or older with infants approximately aged 1 year or less. Women will be screened for study eligibility using the screener questions in Attachments 15b. If they verbally consent (Attachments 15c-15d), they will be offered a self-administered version of the survey that includes the questions to be tested or the survey will be offered in an interview format. Following the survey, women will participate in follow-up interviews to assess survey question quality (Attachment 15b).
Call/Interview Guidelines: Data collection follows a suggested PRAMS interviewing schedule. The protocol suggests up to three mailed surveys be sent to a sampled woman. When jurisdictions deploy the web mode of data collection, letters sent prior to and with the mailed surveys will provide a URL and a QR code with a unique User ID and password allowing the sampled woman to securely access the survey via the web mode. If she does not respond to any of the mailings by completing and returning the paper survey or completing the survey on the web, she is further followed up by telephone, with up to 15 calling attempts made for each viable phone number. Some jurisdictions make calling attempts over the totals suggested by the PRAMS protocol if they have a promising lead. Jurisdictions have some flexibility regarding the mailings. For example, a few jurisdictions elect not to send a third mailing. There is some flexibility in terms of scheduling contact activities; however, the maximum suggested follow-up time for a given batch is 90 days. Jurisdictions also have flexibility regarding frequency and timing of phone calls. All jurisdictions conduct weekday calling, prior to 5pm. Most also conduct weekday evening calling after 5pm. Some jurisdictions also conduct weekend calling on Saturdays and Sundays. During telephone follow-up, mothers not wishing to complete a telephone interview are offered the option of having another survey mailed to them or providing an email address so that the web login information can be emailed to them. The sample batch schedule is provided below (Table B.2-1). The data collection procedures for the surveys to mother with a recent stillbirth follow the same procedures for those of mothers with a recent live birth with some important distinctions. Table B.2-2 summarizes the differences between the two approaches.

Table B.2-1. Timing of PRAMS Data Collection Activities

Action Recommended Time Frame Schedule

1. Mail* pre-letter Day 1 Day 1

2. Mail first questionnaire 3-7 days after pre-letter Day 4-8

3. Mail tickler/reminder 7-10 days after first questionnaire Day 11-18

4. Mail second questionnaire 7-14 days after tickler Day 18 -32

5. Mail third questionnaire 7-14 days after second questionnaire Day 25 - 46

6. Initiate telephone calls 7-14 days after third questionnaire Day 32 - 60

7. End data collection 21-35 days after initiating phone Day 53 - 90

*All mailings (including pre-letters) to mothers with a recent live birth will include the

option to complete by web.

Table B.2-2: Comparison of Data Collection Procedures

	Stillborn Infant	Live Birth

Initial contact	3-6 months after loss	2-6 months after live birth
Pre-letter	Pre-letter will take the form of a hand-written sympathy card on card stock paper. Handwritten signature Purple envelopes are recommended. Postage stamps are affixed on the envelope.	Pre-letter is printed out on regular letterhead paper and regular envelopes. Signature font or hand-written signatures allowed. Provides option to complete survey by web Stamp or metered postage allowed.
Gifts/rewards	Gifts are used. No cash, coupons, or gift cards should be used. Gifts/rewards should be sensitive to the mother’s loss. Recommend sending items that memorializes the baby such as a keychain, charm, pendant, or keepsake. Other gifts like music CD or sachet may also be appropriate. All women should receive the same item.	Gifts and/or incentives are used. No specific guidance by CDC on type of gift/incentive; except that if gift/incentive is an item for baby, there should be a different gift for women whose infants have dies since birth.
Contents of Survey Packet	Cover letter Gift Informed consent Q & A sheet Resource list (for grief counseling and support services) Calendar Survey booklet Return postage paid envelope	Cover letter Gift Informed consent Q & A sheet Resource list Calendar Survey booklet Return postage paid envelope
Mail 1	More personalization of materials including: Handwritten signatures on letter Handwritten address on envelope Postage stamps	Personalization of materials is optional.
Tickler	More personalization of materials including: Handwritten signatures on letter Handwritten address on envelope Postage stamps	Personalization of materials is optional.
Mail 2	More personalization of materials including: Handwritten signatures on letter Handwritten address on envelope Postage stamps	Personalization of materials is optional.
Mail 3	Not used	Recommended
Telephone call attempts	5-8 call attempts to each phone number	15 call attempts to each phone number
Answering machine messages	Allowed, no more than 2 messages to each number.	Allowed, no restrictions on number of messages.
Refusal conversions	Not used	Recommended

*All mailings (including pre-letters) to mothers with a recent live birth will include the option to complete by web

The following conditions apply to phone interviewing:

Change schedules to accommodate holidays and special events
Except for mothers with recent stillbirth or distraught or verbally abusive respondents, eligible persons who initially refuse to be interviewed may be contacted at least one additional time and given the opportunity to be interviewed. Preferably, this second contact will be made by a supervisor or a different interviewer. Jurisdictions determine individually if they will call back these sampled individuals.
Adhere to respondents’ requests for specific callback times whenever possible
Jurisdictions often continue calling beyond the 90-day scheduled time frame if they have failed to achieve site-specific targeted response rates for the batch, or if they have new phone numbers or promising leads on sampled women. However, all follow-up activities must end on the sampled infants’ 9-month birthday, at which point CDC will not accept the responses.

Final Disposition

Sampled women are given a final mail, web, and telephone disposition code when their batch is closed by the jurisdiction. If a clear disposition is not indicated at the time the batch closes (in the case that a mother was never successfully contacted), the PIDS data collection automatically assigns the appropriate disposition code (i.e. non-respondent) when the data release function is activated by the jurisdiction. An interview is considered to be a partial complete if the respondent answers a pre-determined core question that is found approximately 1/5 into the survey and includes response to the respondent birthdate question that is used to validate that the correct women answered the survey. Partially completed surveys are counted as responses.

The final disposition codes are then used to calculate response rates, cooperation rates and refusal rates. The distribution of individual disposition codes and the rates of cooperation, refusal, and response are sent to each jurisdiction when they receive the annual weighted dataset for each calendar year. PRAMS uses standards set by the American Association of Public Opinion Research (AAPOR) to determine response rates.

Procedures to Promote Data Quality and Comparability

In order to maintain consistency across jurisdictions and allow for jurisdiction-to-jurisdiction comparisons, the CDC PRAMS team sets standard protocols for data collection, which all jurisdictions are encouraged to adopt with technical assistance provided by CDC. The following items are included in the PRAMS survey protocol:

All jurisdictions must ask the core questions without modification. Jurisdictions may choose to add standard and/or jurisdiction-added questions to their surveys. Phone interviewers may not offer information to respondents on the meaning of questions, words or phrases beyond the interviewer instructions provided by CDC.
All PRAMS jurisdictions follow the PRAMS Model Protocol as it pertains to sending mailings within the set timeframe indicated in the schedule above and conducting follow-up phone calls for mail (and web) non-respondents. The only element of the data collection activities that is considered optional is the third mailing, even though the overwhelming majority of jurisdictions elect to keep it. Jurisdictions have been encouraged to expand outreach activities to increase response rates. Activities have involved enhancing rather than changing model protocol procedures. Examples include hand delivery of PRAMS survey packets to sampled women at WIC clinics or on tribal reservations. If a PRAMS jurisdiction would like to employ an alternative method to the standard data collection protocol, they must submit their proposal to CDC PRAMS. If it is approved, it must then obtain CDC and local IRB approval.
All PRAMS staff, including phone interviewers, must complete the CDC PRAMS’ Human Subjects Training upon being hired, and annually thereafter.
PRAMS Telephone interviewer training is offered to all interviewers. Systematic, unobtrusive monitoring of 10% of all interviews is a routine part of monthly survey procedures for all interviewers.
Data entry verification is performed by jurisdiction staff on 10% of mail surveys that are entered into PIDS.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Mailing addresses for sampled mothers are obtained from the birth certificate record. These addresses tend to be accurate. All jurisdictions use address correction notification to capture change of address information from the U.S. Postal Service. Phone numbers are available in some jurisdictions on the birth certificate record. Many jurisdictions also have access to health department databases such as newborn screening, WIC (Supplemental Nutritional Program for Women, Infants, and Children), or Medicaid, for obtaining up-to-date contact information to facilitate contacting sampled women.

Sampled mothers are contacted between 2 and 6 months after the birth. Most jurisdictions try to contact women by mail 3 months after birth to reduce the chances of a sampled woman moving to a different address. Jurisdiction staff search both publicly available sources and jurisdiction health department sources in the case of undelivered mailings. The average undelivered mail rate is 6.5%, but with the address and phone number searches, about half of sampled women with an undelivered mail eventually complete a survey.

Telephone follow-up is initiated for mail and web nonrespondents after 3 mailing attempts. Interviewers make a minimum of 15 call attempts per available telephone number varied throughout the day over mornings, afternoons, and evenings and during weekdays and weekends.

A web survey mode of data collection has been designed to give prospective participants another mode for completing the survey. The additional web mode may potentially result in an increase in the response rates overall and better reach populations with lower response rates (e.g., mothers with lower education attainment). Retention of the phone and mail modes of data collection are intended to ensure that there are no negative impacts on response rates including providing the opportunity to complete the survey for sampled women that do not have internet access. Five PRAMS jurisdictions have been implementing the web mode of data collection since May 2022. Based on an examination of key metrics from the initial implementation of the web mode with five jurisdictions (e.g., response rates, item nonresponse) which is described below and in Attachment 2f, all PRAMS jurisdictions will implement the web mode during Phase 9.

As noted in the preceding section, PRAMS uses a number of techniques to deal with nonresponse. The principles and practices of the mixed-mode survey methodology incorporated in PRAMS are based primarily on the Tailored Design Method (TDM) by Don Dillman. The key features of the TDM that have been demonstrated to improve response rates to mail surveys include:

Make multiple and varied contacts. PRAMS methodology includes multiple mailings as described above in the sample batch schedule. Each mailing will offer the option of completing the survey via web. Telephone follow-up is initiated for mail and web nonresponders. Refer to Table B.2-2 for a complete listing of PRAMS mailings and their timing.
Provide a response gift and/or reward. Attachment 5 contains a list of gifts and/or rewards used by each participating PRAMS site. PRAMS grantees have historically conducted experiments to determine what kind of gift or reward is improves survey response as part of the PRAMS protocol.
Develop a respondent-friendly questionnaire. The mail survey booklet includes a colorful, appealing cover image customized by each jurisdiction.
Provide postage-paid return envelopes.
Personalize all correspondences. To personalize mailing letters, names and addresses of samples mothers are printed directly into the letters using mail merge procedures. Where possible, addresses are printed directly onto mailing envelopes.

Additional measures to maximize response rates include providing the interview in languages other than English, creating a protocol designed to convert phone refusals, and alternating times and days of calling attempts. In addition, PRAMS advises jurisdictions to make use of caller ID to inform potential respondents that jurisdiction health departments are making the calls. Interviewers may also leave answering machine messages with their name and contact number on the first call attempt. Sampled mothers are informed of the purpose of the call and the importance of their response early in the introductory script. Experienced interviewers are used for converting phone refusals when respondents initially refuse to take part in the survey. Hard refusals (where potential respondents jurisdiction that they are not interested in completing the interview) are not called back.

Women with a recent stillbirth experienced a recent pregnancy loss. They will still be grieving the loss of their stillborn child. All contact approaches and materials should be carefully developed to be sensitive to the emotional state of these women. Some of the more assertive tactics used in data collection for women who delivered a live birth would not be appropriate for women who have experienced a stillbirth. Sensitivity considerations should be built into all components of the data collection methodology for stillbirth data collection.

PRAMS grantees offer gifts and/or rewards to women who are sampled to participate in the survey. In general, gifts and rewards have been found to be important for encouraging participation in federal surveys. For PRAMS increasing motivation to participate through gifts and/or rewards is particularly salient: this survey focuses on a special population during a limited time period following the birth of an infant when women indicate that participating in even simple activities is constrained by lifestyle changes, financial constraints, childcare duties, and fatigue. Give these constraints, gifts and rewards have been found important for PRAMS and have been demonstrated to improve response rates among subpopulations less likely to respond (including women from racial and ethnic minority groups, women with lower education or lower income) and thereby representativeness of PRAMS sample. Many grantees conduct experiments to determine what kind of gift or reward is most effective in motivating survey response.

Jurisdictions must maintain training for all interviewers involved in PRAMS. Data collectors also participate in monthly or bimonthly conference calls with their CDC program manager to review operations and response rates. Issues related to response rates are discussed on these calls. In addition, a list server of all PRAMS data collectors is available for discussion and dissemination of various approaches to improve response rates.

Although response rates overall for surveys are declining, PRAMS maintains an overall response rate comparable to other surveys. The AAPOR weighted response rates #6 for PRAMS jurisdictions in 2019 ranged from 40% in South Carolina to 81% in Puerto Rico, with a median rate of 57% (for more details, see https://www.cdc.gov/prams/prams-data/researchers.htm#data). We are aware of no other U.S. population-based health surveys using the same methodology as PRAMS. Table B.3-1 provides some comparisons of the PRAMS response rate when compared to other surveys and Table B.3-2 provides a summary of changes in response rates for PRAMS in the last 15 years.

Table B.3-1. Comparisons of Survey Response Rates

Survey	Year(s)	Overall Response Rates
PRAMS¹	2019	56.7%
California Maternal Infant Health Assessment (MIHA)²	2019	61.0%
National Immunization Survey (NIS)³	2020	65.8%
National Survey of Children’s Health (NSCH)⁴	2019	42.4%
Behavioral Risk Factor Surveillance System (BRFSS)⁵	2020	47.9%
National Health Interview Survey (NHIS)⁶	2020	50.9%
National Health and Nutrition Examination Survey (NHANES)⁷	2017- 2018	48.8%
¹PRAMS response rate presented here is the median rate for all sites
²California Maternal Infant Health Assessment (2019)
National Immunization Survey (2020); Age-eligible children with completed household interview and completed health insurance module.
⁴National Survey of Children's Health (2019); the NSCH interview completion rate defined as the proportion of households known to include children that completed all sections of the survey.
⁵https://www.cdc.gov/brfss/annual_data/2020/pdf/2020-response-rates-table-508.pdf
⁶https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2021/srvydesc-508.pdf
⁷National Health and Nutrition Examination Survey (2017-2018); unweighted response rate

Table B.3-2. PRAMS Median Annual Response Rates (%) and by Jurisdiction, 2006-2020

	2006	2007	2008	2009	2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	2020
Annual Median Response	71	70	70	69	67	67	65	63	61	63	60	59	58	57	58
Jurisdiction
Alabama	60	64	60	58	62	59	49	53	61	60	45	55	54	56	55
Alaska	77	76	71	69	65	64	65	69	65	66	61	62	60	58	54
Arizona	N/A¹	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	33	46	46	53
Arkansas	72	74	70	68	68	66	62	61	58	63	56	51	52	56	58
Colorado	70	70	68	70	70	67	67	61	59	63	59	63	62	59	63
Connecticut	N/A	N/A	N/A	N/A	N/A	N/A	N/A	57	60	62	60	61	58	56	63
Delaware	N/A	65	80	73	75	73	70	68	65	64	64	62	59	57	54
District of Columbia	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	42	51	52	59
Florida	N/A	50	57	61	61	62	54	55	48	43	N/A	N/A	42	51	56
Georgia	71	67	68	66	65	67	68	66	47	34	N/A	68	59	61	53
Hawaii	71	74	73	73	72	71	68	70	64	62	N/A	N/A	N/A	56	62
Illinois	75	73	73	71	70	67	66	65	66	66	60	55	61	59	61
Indiana	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	54	51	49	49
Iowa	N/A	N/A	N/A	N/A	N/A	N/A	N/A	72	64	63	63	62	51	50	51
Kansas	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	63	61	63	66
Kentucky	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	59	60	60	60
Louisiana	52	56	52	53	54	57	52	58	59	66	63	66	64	57	57
Maine	74	73	75	71	73	71	70	65	62	58	56	62	57	61	55
Maryland	71	70	73	69	67	65	65	65	66	65	62	57	50	51	50
Massachusetts	N/A	70	72	68	66	69	68	62	60	63	60	62	62	61	60
Michigan	70	65	67	67	70	67	61	60	57	55	55	56	58	56	59
Minnesota	76	75	71	70	66	66	66	60	54	54	48	52	57	55	54
Mississippi	70	54	68	70	64	63	42	27	N/A	N/A	51	53	60	65	60
Missouri	N/A	65	63	67	76	72	73	69	69	66	62	64	57	57	57
Montana	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	62	53	51	53
Nebraska	78	71	74	76	78	73	69	66	60	63	60	54	61	64	66
Nevada	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	41	39	42	43
New Hampshire	N/A	N/A	N/A	N/A	N/A	N/A	N/A	67	64	62	60	55	53	51	52
New Jersey	74	74	72	72	73	72	71	72	72	70	71	70	67	65	65
New Mexico	64	63	61	61	61	67	68	66	66	64	64	64	60	67	62
New York	70	69	71	63	66	68	59	60	61	63	66	56	51	53	49
New York City	70	65	62	64	67	67	67	68	72	72	73	67	65	61	61
North Carolina	60	71	72	63	56	54	51	44	55	51	52	57	53	50	45
North Dakota	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	70	60	59	61
Ohio	72	67	68	67	65	60	63	56	60	58	N/A	N/A	N/A	N/A	N/A
Oklahoma	74	71	71	73	68	67	65	63	62	68	63	58	53	48	40
Oregon	72	67	65	70	75	71	62	62	57	56	52	49	53	69	64
Pennsylvania	N/A	70	70	69	66	69	65	68	69	69	64	64	61	58	60
Puerto Rico	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	62	80	81	81
Rhode Island	73	72	70	71	68	69	65	62	62	52	60	59	58	55	47
South Carolina	70	68	59	59	55	61	48	53	50	45	N/A	46	42	40	43
South Dakota	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	67	64	68	67
Tennessee	N/A	63	70	67	61	61	60	61	60	60	48	48	45	54	56
Texas	54	58	64	67	65	62	59	55	53	56	55	50	44	41	39
Utah	85	81	80	81	81	76	72	66	69	67	65	66	62	73	67
Vermont	83	85	83	83	83	81	79	75	74	70	71	68	69	62	66
Virginia	N/A	57	52	51	54	53	45	45	49	64	60	57	63	56	58
Washington	75	73	79	76	78	74	68	65	60	59	58	61	62	64	64
West Virginia	71	71	72	70	65	70	64	66	63	59	57	55	56	49	50
Wisconsin	N/A	71	66	66	61	67	65	63	60	59	56	59	55	60	64
Wyoming	N/A	70	69	68	67	71	62	62	63	56	63	59	62	55	51

¹N/A: Jurisdiction did not participate in PRAMS during the year

Response rates, cooperation rates, and refusal rates for PRAMS are calculated using standards set by the AAPOR [1]. PRAMS calculates response rates using AAPOR Response Rate #6, cooperation rates using AAPOR Cooperation Rate #2, and AAPOR Refusal Rate #3.

Based on the guidelines of AAPOR, response rate calculations include assumptions of no cases of unknown eligibility among potential respondents that are not interviewed. The eligibility requirements are implemented during the sampling process. The only situations where sampled women are ineligible include cases of adoption or surrogate birth that were not noted on the birth certificate or unusual circumstances where the eligibility requirements were not properly implemented during the sampling process. Since PRAMS utilizes a sampling design with unequal probabilities of selection, weighted response rates are calculated.

4. Tests of Procedures or Methods to be Undertaken

PRAMS protocols have been adapted over time to meet the needs of the data collection process and maximize response rates while minimizing respondent burden. PRAMS continually assesses its methods and procedures through comparisons with industry standards, consultation with PRAMS coordinators and other experts in the field, and real-world experience and feedback from the PRAMS data collectors. PRAMS conducts cognitive testing of new and modified questions from core survey, standard modules, supplemental modules, and call back surveys to ensure that respondents understand the questions correctly and can provide accurate responses. With each questionnaire revision, new or heavily modified questions are tested before inclusion to the core or standard surveys. Existing and unmodified questions are not usually tested again. Field tests are also conducted prior to any questionnaire revision to ensure it is ready for fielding. New, never tested, or substantially revised questions are tested before adoption. Each jurisdiction is responsible for testing their own questions developed for use by their jurisdictions; sites are not required to follow CDC’s procedures (or expectations for) question development and testing. The methods by which questions are adopted are provided in Attachment 4. As this document indicates, Subject Matter Experts from CDC and other federal agencies, jurisdiction health department representatives and survey experts are involved in the process of question development. Many of the questions which are included in the PRAMS align with those from other federal surveys including the National Health Interview Survey (NHIS), the National Adult Tobacco Survey (NATS), the National Health and Nutrition Examination Survey (NHANES), the National Immunization Survey (NIS) and others. The use of identical or similar questions is advantageous in that it allows researchers to make comparisons across different samples, different geographic areas or over time. PRAMS Phase 9 questions were cognitively tested by the Collaborating Center for Questionnaire Design and Evaluation Research hosted by NCHS, OMB No. 0920-0222, Exp. 08/31/2021 (Attachment 17). Cognitive testing of new or substantially modified supplemental module or call back survey question will be conducted under an appropriate generic clearance. Additional field testing of new or substantially modified questions will be conducted using PRAMS field testing methodology (Attachment 15a). The results of the PRAMS Phase 9 questions field testing can be found in Attachment 18.

To further the use of improved information technology, CDC, through a contractor, has developed a web mode in PIDS that is integrated alongside the mail and telephone features. The intent is for participants to have the option of an additional mode by which to complete the PRAMS survey. Thus, once the web mode is available, respondents can complete the survey by mail, phone or web depending on their own needs and preferences. The web mode may better reach some under-represented populations and result in an increase in the response rate overall and improved representativeness of survey respondents.

In the spring of 2022, OMB approved implementation of the web survey mode in five sites (Maryland, Puerto Rico, South Carolina, Virginia, and Wyoming). Web versions of the Phase 8 survey were programmed in PIDS for these five early adopter sites (Attachment 2a-2e) and deployed in May 2022. Prior to implementation, the web mode of data collection completed two rounds of User Acceptance Testing [UAT] with the five early adopter sites (MD, VA, PR, WY, SC). UAT was conducted by staff at CDC and the five grantee sites to test various use cases (e.g., ineligible participant, completed survey, break-off, refusal), optimize the display across devices (mobile phone, tablet, laptop, desktop), verify the survey flow and skip patterns, and provide feedback on the usability of the web survey experience.

All five early adopter sites began implementing the web mode with the May 2022 batch of mothers who were invited to participate. All potential participants within those sites were offered the opportunity to complete the survey by web, mail or telephone interview. A pre-survey letter and up to three mailed surveys are sent to all potential participants. The pre-survey letter introduces PRAMS and informs of the option to complete the survey by paper, web, or telephone. The letters sent prior to and with the mailed (paper) surveys include a URL and a QR code with a unique User ID and password, allowing the sampled mother to securely access the web mode survey. If no response by web or mail is received, up to 15 call attempts are made for each viable phone number. Sampled mothers contacted by phone are provided with the option to complete the survey as an interview over the phone or by mail or web.

The May 2022 batch completed the 90-day data collection cycle at the end of July 2022. Analyses comparing metrics for the May 2022 web batch with the April 2022 batch (for which participants did not have the web mode option) examined changes in response rates, response rates by subpopulation, data quality (measured by item non- response), and grantee resources and costs. Results are summarized below with additional details provided in Attachment 2f.

For the five sites overall, the average difference in response rates was an increase of 2.2 percentage points after the web mode was added. Four of the five sites experienced increases in response rates after adding the web mode ranging from 1 to 7 percentage points. One site experienced a decline of 7 percentage points. That site initiated the social determinants of health supplement in the same month as the web mode of data collection was added; these additional questions may have contributed to their response rate decline. None of these changes was statistically significant.

Response by subpopulation was examined by age, race, ethnicity, and education. For all categories of age, race, ethnicity, and education, response rates increased after the addition of the web mode with the exception of the less than 20-year-old group, though none of these changes were statistically significant. This youngest age group had a small sample size even across the five sites combined making it possible that the decrease reflects random survey response fluctuations. For subpopulations that are known to have lower response rates, like Black women and those with less than a high school education, response rates were 2.2 and 9.8 percentage points higher, respectively, after the addition of the web mode. This demonstrates the utility of web mode data collection for potentially improving the representativeness of PRAMS respondents.

Six questions on the PRAMS survey were used to assess the degree of item nonresponse as a measure of data quality. Item nonresponse rates for the web mode were consistently lower than corresponding rates for mail and phone modes. For example, mail and phone respondents had item nonresponse rates 4.5 and 20.5 times higher, respectively, than web respondents for the income question.

Mailing and associated postage costs also were tracked. All sites experienced a reduction in postage costs after implementing the web mode that resulted from fewer follow-up mailings being needed. The reduction ranged from 4% to 13% of the pre-web costs.

Early adopter sites also tracked the staff time required to conduct data collection activities. Four of the five sites reported a reduction in staff hours after implementation of the web mode. The reduction ranged from 41% to 6% of the pre-web staff hours. One site experienced a 17% increase in staff hours. The site that experienced an increase in staff hours explained that, because web mode respondents have a choice of reward, staff had to contact each respondent individually to confirm reward selection. Staff did not have to contact mail or phone respondents since the mailed surveys included a card for respondents to select their choice of reward and return with the survey and phone respondents are asked of their preference of reward at the end of the interview. The other four sites either provide the same reward to all participants or do not provide a reward for web respondents and so did not need to follow-up.

And lastly, the number of queries about the PRAMS survey from sampled mothers were also tracked as a measure of ease of use of the web mode. Over all five sites, there were 8 queries in the pre-web batch and 8 queries in the post-web batch. The consistent number of queries implies that the instructions and interface for the web mode are not causing confusion, and sampled mothers can access and navigate the web system without additional technical support.

In summary, across each metric examined, the post-web metrics indicated modest overall improvement relative to their pre-web counterparts. After the web mode was implemented, there was, on average, a 2 percentage point increase in response rates, improved response rates for all categories of race, ethnicity, and education level examined, reduced item nonresponse as compared to mail and telephone modes, reduced postage costs by 4% to 13%, reduced staff hours expended by 6% to 41% in all but one site, and no change in the number of technical support queries from sampled mothers. CDC continues to monitor response rates and other performance measures by mode of data collection for the five early adopter sites to identify strategies and technical assistance needs for jurisdictions to address response rates when implementing the web mode. Based on these findings from Phase 8 web mode of data collection, we would like to add the web mode to Phase 9 data collection in April 2023. Jurisdictions will be onboarded for web mode in phases so each jurisdiction will receive adequate technical assistance should they need it at the start of implementation. By September 2023, all 50 jurisdictions will be offering web mode, in addition to mail and telephone data collection.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

CDC personnel are responsible for all statistical aspects of the PRAMS including data analyses and reporting. The following staff members are primarily responsible for PRAMS data reporting.

Name	Title	Phone	E-mail
Cynthia Cassell	Team lead, PRAMS	678-429-6444	ivv7@cdc.gov
Brenda Bauman	Team Senior Scientist	770-488-3756	cmo9@cdc.gov
Holly Shulman	Statistician	610-690-0861	hbs1@cdc.gov
Aspy Taraporewalla	IT Project Manager	770-488-6222	est3@cdc.gov
Ruben Smith	Statistician	770-488-6567	eyb4@cdc.gov
Stacey Hurst	Data Manager	770-488-4641	svh4@cdc.gov

In the past, the following external consultants have provided input on PRAMS methodology and the analysis of PRAMS data:

Name	Affiliation	Area of Consultation
Don Dillman, PhD	Washington State University	Survey methodology
Donna Brogan, PhD	Emory University	Analysis of complex survey data

References

1. The American Association for Public Opinion Research. 2015. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 8th edition. AAPOR.
http://www.aapor.org/AAPORKentico/AAPOR_Main/media/publications/Standard-Definitions2015_8theditionwithchanges_April2015_logo.pdf

2. The Council of American Survey Research Organizations. 2013. Code of standards and ethics for market, opinion, and social research http://c.ymcdn.com/sites/ www.casro.org/resource/resmgr/code/september_2013_revised_code.pdf?hhSearchTerms=%22casro+and+response+and+rate%22

3. Mail and Internet surveys: The tailored design method--2007 Update with new Internet, visual, and mixed-mode guide. Dillman DA. John Wiley & Sons. 2007.

1PRAMS is a joint surveillance effort between CDC and participating jurisdictions, funded by cooperative agreement DP21-001. This Notice of Funding Opportunity (NOFO) was an open competition and is posted on grants.gov. The jurisdictions included represents those that were funded through this process.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	cww6
File Modified	0000-00-00
File Created	2024-11-24