Pregnancy Risk Assessment Monitoring System (PRAMS)
Existing Collection in Use without an OMB Control Number
Supporting Statement
Part B
October 21, 2019
Contact Person:
Leslie Harrison, MPH
Telephone: 770-488-6335
Fax: 770-488-6291
E-mail: lfl0@cdc.gov
Division of Reproductive Health
National Center for Chronic Disease Prevention and Health Promotion
Centers for Disease Control and Prevention
Atlanta, Georgia
List of Attachments………………………………………………………………………………………………….3
B. Collection of Information Employing Statistical Methods 5
1. Respondent Universe and Sampling Methods 5
2. Procedures for the Collection of Information 8
Content and Construction of the PRAMS Questionnaire(s) 12
Procedures to Promote Data Quality and Comparability 18
3. Methods to Maximize Response Rates and Deal with Nonresponse 19
4. Tests of Procedures or Methods to be Undertaken 22
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 23
Attachments
Attachment 1a. Authorizing Legislation-Public Health Service Act
Attachment 1b. Authorizing Legislation-The Safe Motherhood Act for Research and Treatment
Attachment 1c. Authorizing Legislation-The Prematurity Research Expansion and Education for Mothers who deliver Infants Early (PREEMIE) Act
Attachment 2a. 60 day FRN
Attachment 2b. Program Response to Comments 60 day FRN
Attachment 3. PRAMS Questionnaire Development Process
Attachment 4. PRAMS Response Incentives and Rewards by Jurisdiction
Attachment 5. Privacy Impact Assessment
Attachment 6. PRAMS Phase 8 Questionnaire Topic Reference
Attachment 7a. PRAMS Livebirth Core Phase 8 Mail Questionnaire (English)
Attachment 7b. PRAMS Livebirth Core Phase 8 Mail Questionnaire (Spanish)
Attachment 7c. PRAMS Livebirth Core Phase 8 Phone Questionnaire (English)
Attachment 7d. PRAMS Livebirth Core Phase 8 Phone Questionnaire (Spanish)
Attachment 8a. PRAMS Stillbirth Mail Questionnaire (English)
Attachment 8b. PRAMS Stillbirth Mail Questionnaire (Spanish)
Attachment 8c. PRAMS Stillbirth Phone Questionnaire (English)
Attachment 8d. PRAMS Stillbirth Phone Questionnaire (Spanish)
Attachment 9a. PRAMS Livebirth Standard Phase 8 Mail Modules (English)
Attachment 9b. PRAMS Livebirth Standard Phase 8 Mail Modules (Spanish)
Attachment 9c. PRAMS Livebirth Standard Phase 8 Phone Modules (English)
Attachment 9d. PRAMS Livebirth Standard Phase 8 Phone Modules (Spanish)
Attachment 10a. Family History of Breast and Ovarian Cancer Supplement
Attachment 10b. Prescription and Illicit Opioid Use Supplement
Attachment 10c. Disabilities Supplement
Attachment 11a. Opioid Call Back Survey (English)
Attachment 11b. Opioid Call Back Survey (Spanish)
Attachment 12a. PRAMS Livebirth Introduction and Informed Consent Mail (English)
Attachment 12b. PRAMS Livebirth Introduction and Informed Consent Mail (Spanish)
Attachment 12c. PRAMS Stillbirth Introduction and Informed Consent Mail (English)
Attachment 12d. PRAMS Stillbirth Introduction and Informed Consent Mail (Spanish)
Attachment 12e. PRAMS Livebirth Introduction and Informed Consent Phone (English)
Attachment 12f. PRAMS Livebirth Introduction and Informed Consent Phone (Spanish)
Attachment 12g. PRAMS Stillbirth Introduction and Informed Consent Phone (English)
Attachment 12h. PRAMS Stillbirth Introduction and Informed Consent Phone (Spanish)
Attachment 13a. Call Back Survey Opt Out Recontact Mail (English)
Attachment 13b. Call Back Survey Opt Out Recontact Mail (Spanish)
Attachment 13c. Call Back Survey Opt Out Recontact Phone (English)
Attachment 13d. Call Back Survey Opt Out Recontact Phone (Spanish)
Attachment 14. IRB Approval
Attachment 15. PRAMS Livebirth Respondent Counts by Jurisdiction
Attachment 16. Opioid Call Back Survey Methodology
Attachment 17. Field Testing Methodology
Attachement 18. Field Testing Questionniare
Attachment 19a. PRAMS Field Testing Questionniare Informed Consent (English)
Attachment 19b. PRAMS Field Testing Questionniare Informed Consent (Spanish)
The Pregnancy Risk Assessment Monitoring System (PRAMS) is a collaborative project of the Centers for Disease Control and Prevention (CDC) and U.S. states. Beginning in May 2016, PRAMS is funded in 51 sites and covers 83% of all live births in the United States. Sites that collect data on women with recent livebirths now include 47 states (excluding Idaho, Ohio, and California), New York City, Washington, DC, Puerto Rico, and Great Plains Tribal Chairman’s Health Board (henceforth referred to as ‘states’). Utah is the only site currently funded to collect data on women with a recent stillbirth. The PRAMS is a coordinated series of self-administered questionnaires and phone interviews that collect information about maternal experiences and behaviors before and during pregnancy and after pregnancy. Respondents are mothers who have recently given birth to a live born or stillborn infant. Information collection is usually conducted continuously; however, a point in time data collection is allowable.
The PRAMS is administered through cooperative agreements with state health departments. A representative sample of respondents is drawn monthly by each state. Each state administers a state-tailored questionnaire that consists of (1) a set of core questions administered by all states and (2) standard module questions selected by the states as desired. The state-tailored questionnaires and samples are designed individually by each state with technical assistance provided by the CDC. States may be funded to include supplemental modules for emerging issues. In addition, call back surveys may be implemented to gather additional information on post-pregnancy experiences and infant and toddler health. Women who respond to the initial PRAMS survey may be re-contacted (opt-out consent process used) at a later time (beyond 6 months post-birth) to collect additional information about post-pregnancy experiences and infant and toddler health.
Field testing will be conducted to identify problems with new or substantively revised questions. A convenience sample of women with young infants who are approximately one year of age or less will be recruited in clinics or doctor’s offices to answer a short survey and provide feedback on the quality of questions. Upon verbal consent, women will be offered a self-administered version of the field testing survey or the survey will be offered in an interview format. Following the survey, women will be asked to provide feedback on the quality of proposed questions.
Respondent Universe
The target population for PRAMS data collection is mothers who have recently given birth to a live or stillborn infant. An eligible mother is defined as a mother who resides in that state and gave birth within that state. The following are non-eligible mothers: mothers who are not state residents, mothers who gave birth outside of the state, and mothers who gave birth more than six months after the date of sampling.
Proxy interviews are not conducted within the PRAMS.
State-tailored Samples
An independent sample is drawn monthly by each state. The size of each state sample is pre-determined in collaboration with the state health department and CDC. States may subdivide their state by geographic region/geostrata (such as public health districts, counties or groups of counties). States may also oversample population groups within their sample. To ensure an adequate number of responses for analysis purposes, most states must conduct at least 400 questionnaires/interviews annually for each analysis stratum, with most states choosing between 2 and 6 strata per state.
See Attachment 15 for the estimated size of the annual PRAMS livebirth respondents by state. The state of Utah is the only state currently funded for stillbirth surveillance. It is estimated that there will be approximately 160 responses for the stillbirth surveillance in Utah.
Sampling Frame
To provide rapid and flexible access to respondents and contain costs, PRAMS data collection is conducted through self-administered questionnaires and telephone interviews. A sample record is one birth certificate or fetal death record originating that month. The PRAMS sample is randomly selected from vital records within each jurisdiction. Each state randomly selects a random sample of mothers from within each sampling stratum, at a sampling rate predetermined by the state and CDC.
Each state obtains the list of birth certificates or fetal death records from their Vital Records department on a monthly basis. Once the sample has been drawn from the eligible list, each mother is mailed the PRAMS questionnaire. If the mother has not returned the completed questionnaire after three such mailings, efforts are made to contact and administer the PRAMS questionnaire via phone.
In addition, PRAMS may employ unique sampling strategies for special research questions. For example, the Maternal and Child Health Bureau (MCHB) in the Health Resources and Services Administration (HRSA) used the PRAMS platform to help in the data collection efforts for a national evaluation of the transformed Healthy Start program. MCHB/HRSA randomly selected 15 Healthy Start grantee sites to participate for the one-time oversampling (oversampling of Healthy Start participants for 2017- 2018 only). The goal of the evaluation is to determine the effect of the Healthy Start program on changes in participant-level characteristics (e.g., health services utilization, preventive health behaviors, health outcomes). In these states, Healthy Start participants were identified from vital records using personal identifiers provided by Healthy Start grantees (e.g., name, date of birth, date of delivery); PRAMS programs then sampled all identified Healthy Start participants for comparison with non-Healthy Start Participants. Data collection is complete for this special project, weighting and analyses are ongoing. For the ongoing opioid call back surveys implemented in October 2019, each participating state has designed a sampling plan to oversample women in areas of the state where there is based on metrics of burden selected by the state (e.g., opioid overdose or neonatal abstinence syndrome rates) (Attachment 16).
Table B.1-1. Summary of Similarities and Differences in Sampling by State
Similarities |
Differences |
|
|
Each participating state draws a stratified systematic sample of every month from a frame of eligible birth certificates. Most states oversample low weight births. Many states stratify by mother’s race or ethnicity as well. Typically, the annual sample is large enough for statewide estimates within 3.5% at 95% confidence. Estimated proportions within strata are slightly less precise (typically, they are estimated within 5% at 95% confidence).
Mothers’ responses are linked to extracted birth certificate data items for weighting. The availability of this information for all sampled women, whether they responded or not, is key to deriving nonresponse weights.
For each respondent, the initial sampling weight is the reciprocal of the sampling fraction applied to the stratum. Sampling fractions in PRAMS range from 1 in 1 (e.g. for the very low birth weight strata in small states) to about 1 in 211 (e.g. for the normal birth weight, nonminority strata in populous states). Corresponding sampling weights, then, would range from 1 to 211.
Nonresponse adjustment factors attempt to compensate for the tendency of women having certain characteristics (e.g. unmarried) to respond at lower rates than women without those characteristics. The rationale for applying nonresponse weights is the assumption that non respondents would have provided similar answers, on average, to respondents’ answers for that stratum and adjustment category. There are select characteristics that are evaluated for non-response adjustment. These include race/ethnicity, age, education, marital status, county of residence, hospital size, timing of prenatal care, infant birth weight, method of payment for delivery, WIC status, and parity. Where multivariate analysis shows that select characteristics affect the propensity to respond in a stratum, the adjustment factor is the ratio of the sample size in that category to the number of respondents in the category. So that cells with few respondents are not distorted by a few women’s answers, small categories are collapsed until each cell contains at least 25 respondents. The magnitude of the adjustment for nonresponse depends on the response rate for a category. If 80% (or 4/5) of the women in a category respond, the nonresponse weight is 1.25 (or 5/4). Categories with lower response rates have higher nonresponse weights.
Frame omission studies are carried out to look for problems that occur during frame construction. The frame noncoverage weights are derived by comparing frame files for a year of births to the final calendar year birth file that states provided to CDC. Omitted records are usually due to late processing and are evenly scattered across the state, but sometimes they are clustered by hospitals or counties or even times of the year. The effect of the noncoverage weights is to bring totals estimated from sample data in line with known totals from the birth file. The magnitude of noncoverage is small (typically from 1% to 5%), so the adjustment factor for noncoverage is not much greater than 1.
Multiplying together the sampling, nonresponse, and noncoverage components of the weight yields the analysis weight. Analyzing PRAMS data requires software that takes into account the complex sampling designs that states employ. Such software utilizes first-order Taylor series approximations to calculate appropriate standard errors for the estimates it produces.
A detailed description of PRAMS methodology, including standardized data collection methodology, questionnaire and weighting process is available on the PRAMS website at https://www.cdc.gov/prams/methodology.htm#n4
Procedures for collection of information follow common protocols for sampling and questionnaire administration to produce a coordinated series of state-tailored surveys with a common reference set of questions (divided into core questions and standard module questions). Some flexibility in the design of the sampling stratification and content of each state-tailored questionnaire and operations management is allowed within parameters established by the PRAMS cooperative agreement. Participants in the design and implementation of the data collection process include CDC, state health departments, and data collection contractors.
Summary of Steps, Roles, and Responsibilities
CDC PRAMS revises the PRAMS questionnaire in phases every 3-5 years. The process starts with solicitation of new topics from state and other partners. Representatives from the PRAMS State Working Group provide input to CDC on newly proposed question topics. Detailed information about the process is provided in Attachment 3 (PRAMS Questionnaire Development Process). All states have the opportunity to vote regarding acceptance of newly proposed questions and content of the core questionnaire. Once the core questionnaire is established, all states must use the livebirth core questions unchanged for the entire phase of data collection (Attachments 7a-7d). Questions that are approved, but not added to the core, become part of the standard modules list (Attachments 9a-9d). At the beginning of each Phase, states select their standard modules and may also develop state-specific questions based on state priorities and input from their state steering committees that may consist of internal and external partners such as those from academia, non-profits, health care system, and medical associations. Once the questions are finalized, there are no changes for the length of the questionnaire phase (approximately 3-5 years). A similar process was used to develop the stillbirth questionnaire (Attachments 8a-8d); however, standard module questions are not added as part of the stillbirth survey.
Periodically to respond to emerging issues, CDC or other federal partners fund states to add supplemental modules to the survey mid-questionnaire phase as an insert to the end of the current survey. Currently funded supplemental modules for 2019 include Family History of Breast and Ovarian Cancer, disabilities and prescription and illicit opioids (Attachments 10a-10c). States may also choose on their own to add questions as a supplement to their survey to address state-driven priorities.
Periodically to respond to emerging issues, CDC or other federal partners fund states to do call back surveys. Call back surveys may be implemented to gather additional information on post-pregnancy experiences and infant and toddler health. The currently planned call back survey includes topics such as opioid misuse and access to medication assisted therapy, experiences with respectful care, postpartum care, rapid repeat pregnancy, infant feeding practices, infant health and social services such as well child visit attendance, home visitation, developmental delays, and social supports (Attachments 11a-11b). States may also choose to perform their own call back surveys to address state-driven priorities.
Prior to adding new questions or substantively revising existing ones, PRAMS questions are cognitively tested by the Collaborating Center for Questionnaire Design and Evaluation Research hosted by NCHS, OMB No. 0920-0222, Exp. 08/31/2021. PRAMS also conducts field testing approximately three times per year to identify issues with and obtain feedback on the quality of the questions. No more than 50 women with infants who are about 1 year old or less will be recruited for field testing at clinics or doctor’s offices. Women will be read an informed consent script (Attachments 19a-19b); if they verbally consent, they will be offered a self-administered version of the field testing survey or the survey will be offered in an interview format. Following the survey, women will be asked to provide feedback on the quality of questions (Attachment 18).
CDC produces the mail survey print files, telephone survey hard copy files, and all data processing documentation. Because states select standard modules, as well as state-added questions, the final questionnaire produced for each state is unique to that state, although all states must include questions from the core.
PRAMS grantee staff send out mailings and conduct telephone interviews. Some state PRAMS projects have contractors who conduct phone and/or mail operations. CDC provides the data collection software called PRAMS Integrated Data Collection System (PIDS). PIDS is used to schedule and track data collection activities; record data on mail and telephone results; manage call attempts for telephone interviews; and record survey responses and extraneous comments provided by mothers. PIDS includes a component for data entry of mail survey data and Computer-Assisted Telephone Interviewing (CATI) for telephone interviews. All states must use PIDS for PRAMS operations, including the CATI software.
CDC works with each state to determine its sample size and sampling stratification. State PRAMS staff coordinate with their state Vital Records Department to create the sampling program and implement it on a monthly basis. Monthly operations span across approximately 95 days. The timeline can be customized by the state as long as operations for a batch are completed within the 95 day limit. This time frame allows for approximately 6 weeks of mail follow-up and 6 weeks of phone follow-up for each monthly batch. After phone follow-up, states complete data entry for mail responses into PIDS and conduct quality assurance and validation checks before submitting data to CDC.
PRAMS awardees are responsible for field operations and determine how their data will be collected within PRAMS guidelines. States may collect data using in-house staff or hire vendors using Request for Proposal (RFP) procedures, or contract with universities. Data collectors must develop and maintain procedures to ensure respondents’ privacy, assure and document the quality of the interviewing process, and supervise and monitor the interviewers.
States release de-identified data files to CDC on a monthly basis for cleaning and weighting. CDC returns clean, weighted data files to the state of origin for its use. There is no public use PRAMS dataset. State data that meets the response threshold set by CDC and participating states (55% beginning in 2015) are released for use by external researchers. Researchers may submit a request/application for data following instructions found on the PRAMS web site http://www.cdc.gov/prams/researchers.htm
Maternal and child health indicators derived from PRAMS data are available in aggregate and by state https://www.cdc.gov/prams/prams-data/mch-indicators.html
The PRAMS questionnaire is comprised of a core which includes questions asked of all respondents, standard module questions which are preselected at the beginning of each phase by the states, and state-added questions which are developed by states. Supplemental modules for emerging issues may be added to the survey mid-questionnaire phase as an insert to the end of the current survey. Call back surveys may be implemented to gather additional information on post-pregnancy experiences and infant and toddler health. States may also choose to perform their own supplements and call back surveys to address state-driven priorities.
Attachment 6 provides a topic reference list for all currently approved questions. All questions included in the PRAMS core survey, standard modules, supplemental modules and call back survey are cognitively tested, and field tested prior to inclusion in the questionnaire. States are responsible for conducting field testing on state-added questions. State-added questions that have been fielded for several years and successfully evaluated may be promoted to the standard modules without additional cognitive testing by CDC.
Core Questions: This portion of the questionnaire must be asked by all sites. (Attachment 7a-7d) It remains fixed for all states across the entire questionnaire phase. During the questionnaire revision process that occurs every three years, the core is re-evaluated and adjusted to include questions about emerging health issues. Questions that are demoted from the core after a questionnaire revision become part of the standard module list.
Standard Modules: Sets of standardized questions on various topics that each state may select are available as standard modules (Attachment 9a-9d). Once selected, states are encouraged to use the standard module in its entirety. However, states may drop answer options if not all responses are relevant in their context. If major modifications to a standard question are requested, it becomes a state-added question.
State-added Questions: States may choose to gather data on additional topics related to their specific health priorities using extra questions they choose to add to their questionnaire. States are required to field test questions or provide evaluation evidence that they have been successfully used by another state or survey with a similar population. The CDC reviews and makes suggestions on the questions; however, states ultimately decide on the final state-added questions.
Supplemental Modules: States may be funded to rapidly implement supplemental modules developed mid-Phase to address emerging issues (planned supplemental modules for 2019 include family history of breast and ovarian cancer, prescription and illicit opioid use, and disabilities) (Attachments 10a-10c). If states elect to add additional standard or supplemental modules to the survey mid-phase to address state-specific emerging needs and priorities, questions are added as attached leaflets to the mail survey or added to the end of questionnaire for the phone collection mode.
Call back Surveys: States may be funded to implement call back surveys to gather additional information on post-pregnancy experiences and infant and toddler health. Women who respond to the PRAMS survey may be re-contacted (opt-out consent process used) at a later time (beyond 6 months post-birth) to collect additional information about post-pregnancy experiences and infant and toddler health. The opioid call back survey was implemented in October 2019 in areas with a high burden of opioid overdose deaths and include topics such as opioid misuse and access to medication assisted therapy, experiences with respectful care, postpartum care, rapid repeat pregnancy, infant feeding practices, infant health and social services such as well child visit attendance, home visitation, developmental delays, and social supports (Attachments 11a-11b).
Field Testing: New questions may be added, or existing questions may be substantively revised, prior to a new phase of the core questionnaire and standard modules or with the addition of supplemental modules and/or call back surveys. States participating in the field testing of questions will obtain the permission from the directors of relevant Health Department Clinics and private pediatrician’s offices to recruit women with infants approximately aged 1 year or less in their office waiting rooms. Each woman will be read a consent script that specifies that her participation is voluntary (Attachment 19a-19b). English and Spanish mail and phone versions of the new or substantively revised questions would be randomly distributed to the 50 participating women. Respondents will be asked to answer a short survey and provide feedback on the quality of questions (Attachment 18).
Data collection follows a suggested PRAMS interviewing schedule. The protocol suggests up to 3 mailed surveys be sent to a sampled woman. If she does not respond to any of the mailings, up to 15 calling attempts for each viable phone number are made. Some states make calling attempts over the totals suggested by the PRAMS protocol if they have a promising lead. States have some flexibility regarding the mailings. For example, a few states elect not to send a third mailing. There is some flexibility in terms of scheduling contact activities; however, the maximum suggested follow-up time for a give batch is 95 days. States also have flexibility regarding phone calls. All states conduct weekday calling, prior to 5pm. Most also conduct weekday evening calling after 5pm. Some states also conduct weekend calling on Saturdays and Sundays. The sample batch schedule is provided below (Table B.2-1). The data collection procedures for the surveys to mother with a recent stillbirth follow the same procedures for those of mothers with a recent live birth with some important distinction. Table B.2-2 summarizes the differences between the two approaches.
Table B.2-1. Timing of PRAMS Data Collection Activities
Action Recommended Time Frame Schedule
1. Mail preletter Day 1 Day 1
2. Mail first questionnaire 3-7 days after preletter Day 4-8
3. Mail tickler 7-10 days after first questionnaire Day 11-18
4. Mail second questionnaire 7-14 days after tickler Day 18 -32
5. Mail third questionnaire 7-14 days after second questionnaire Day 25 - 46
6. Initiate telephone calls 7-14 days after third questionnaire Day 32 - 60
7. End data collection 21-35 days after initiating phone Day 53 - 95
Table B.2-2: Comparison of Data Collection Procedures
|
Stillborn Infant |
Live Birth |
|
|
|
Initial contact |
3-6 months after loss |
2-6 months after live birth |
Preletter |
|
|
Incentives/rewards |
|
|
Contents of Survey Packet |
|
|
Mail 1 |
More personalization of materials including:
|
Personalization of materials is optional. |
Tickler |
More personalization of materials including:
|
Personalization of materials is optional. |
Mail 2 |
More personalization of materials including:
|
Personalization of materials is optional. |
Mail 3 |
Not used |
Recommended |
Telephone call attempts |
5-8 call attempts to each phone number |
15 call attempts to each phone number |
Answering machine messages |
Allowed, no more than 2 messages to each number. |
Allowed, no restrictions on number of messages. |
Refusal conversions |
Not used |
Recommended |
The following conditions apply to phone interviewing:
Change schedules to accommodate holidays and special events
Except for mothers with recent stillbirth or distraught or verbally abusive respondents, eligible persons who initially refuse to be interviewed may be contacted at least one additional time and given the opportunity to be interviewed. Preferably, this second contact will be made by a supervisor or a different interviewer. States determine individually if they will call back these sampled individuals.
Adhere to respondents’ requests for specific callback times whenever possible
States often continue calling beyond the 95 day scheduled time frame if they have failed to achieve the 65% response rate threshold for the batch, or if they have new phone numbers or promising leads on sampled women. However, all follow-up activities must end on the sampled infants’ 9-month birthday, at which point CDC will not accept the responses.
Final Disposition
Sampled women are given a final mail and telephone disposition code when their batch is closed by the state. If a clear disposition is not indicated at the time the batch closes (in the case that a mother was never successfully contacted), the PIDS data collection automatically assigns the appropriate code when the data release function is activated by the state. An interview is considered to be a partial complete if the respondent answers a pre-determined core question that is found approximately 1/5 into the survey and includes response to the respondent birthdate question that is used to validate that the correct women answered the survey. Partially completed surveys are counted as responses.
The final disposition codes are then used to calculate response rates, cooperation rates and refusal rates. The distribution of individual disposition codes and the rates of cooperation, refusal, and response are sent to each state when they receive the annual weighted dataset for each calendar year. PRAMS uses standards set by the American Association of Public Opinion Research (AAPOR) to determine response rates.
In order to maintain consistency across states and allow for state-to-state comparisons, the CDC PRAMS Team sets standard protocols for data collection, which all states are encouraged to adopt with technical assistance provided by CDC. The following items are included in the PRAMS survey protocol:
All states must ask the core questions without modification. States may choose to add standard and/or state-added questions to their surveys. Phone interviewers may not offer information to respondents on the meaning of questions, words or phrases beyond the interviewer instructions provided by CDC.
All PRAMS states follow the PRAMS Model Protocol as it pertains to sending mailings within the set timeframe indicated in the schedule above and conducting follow-up phone calls for mail non-respondents. The only element of the data collection activities that is considered optional is the third mailing, even though the overwhelming majority of states elect to keep it. States have been encouraged to expand outreach activities to increase response rates. Activities have involved enhancing rather than changing model protocol procedures. Examples include hand delivery of PRAMS survey packets to sampled women at WIC clinics or on tribal reservations. If a PRAMS state would like to employ an alternative method to the standard data collection protocol, they must submit their proposal to CDC PRAMS. If it is approved, it must then obtain CDC and local IRB approval.
All PRAMS staff, including phone interviewers, must complete the CDC PRAMS’ Human Subjects Training upon being hired, and annually thereafter. Data entry verification is performed by state staff on 10% of mail surveys that are entered into PIDS.
PRAMS Telephone interviewer training is offered to all interviewers. Systematic, unobtrusive monitoring of 10% of all interviews is a routine part of monthly survey procedures for all interviewers.
Mailing addresses for sampled mothers are obtained from the birth certificate record. These addresses tend to be accurate. All states use address correction notification to capture change of address information from the U.S. Postal Service. Phone numbers are available in some states on the birth certificate record. Many states also have access to health department databases such as newborn screening, WIC (Supplemental Nutritional Program for Women, Infants, and Children), or Medicaid, for obtaining up-to-date contact information to facilitate contacting sampled women.
Sampled mothers can be contacted between 2 and 6 months after the birth. Most states try to contact women 2 to 3 months after birth to reduce the chances of a sampled woman moving to a different address. State staff search both publicly available sources and state health department sources in the case of undelivered mailings. The undelivered mail rate is 6.5%, but with the address and phone number searches, about half of sampled women with an undelivered mail eventually complete a survey.
Telephone follow-up is initiated for mail nonrespondents after 3 mailing attempts. Interviewers make a minimum of 15 call attempts per available telephone number varied over mornings, afternoons and evenings, weekdays and weekends.
As noted in the preceding section, PRAMS uses a number of techniques to deal with nonresponse. The principles and practices of the mixed-mode survey methodology incorporated in PRAMS are based primarily on the Tailored Design Method (TDM) by Don Dillman. The key features of the TDM that have been demonstrated to improve response rates to mail surveys include:
Make multiple and varied contacts. PRAMS methodology includes multiple mailings as described above in the sample batch schedule. Telephone follow-up is initiated for mail nonresponders. Refer to Table B.2-1 for a complete listing of PRAMS mailings and their timing.
Provide a response incentive and/or reward. Attachment 4 contains a list of incentives and rewards used by each participating PRAMS site. PRAMS grantees have historically conducted experiments to determine what kind of gift or reward is most effective in motivating survey response as part of the PRAMS protocol.
Develop a respondent-friendly questionnaire. The mail survey booklet includes a colorful, appealing cover image customized by each state.
Provide postage paid return envelopes.
Personalize all correspondences. To personalize mailing letters, names and addresses of samples mothers are printed directly into the letters using mail merge procedures. Where possible, addresses are printed directly onto mailing envelopes.
Additional measures to maximize response rates include providing the interview in languages other than English, creating a callback protocol designed to convert phone refusals, and alternating times and days of calling attempts. In addition, PRAMS advises states to make use of caller ID to inform potential respondents that state health departments are making the calls. Interviewers may also leave answering machine messages with their name and contact number on the first call attempt. Sampled mothers are informed of the purpose of the call and the importance of their response early in the introductory script. Experienced interviewers are used for callbacks when respondents initially refuse to take part in the survey. Hard refusals (where potential respondents state that they are not interested in completing the interview) are not called back.
Women with a recent stillbirth experienced a recent pregnancy loss. They will still be grieving the loss of their stillborn child. All contact approaches and materials should be carefully developed to be sensitive to the emotional state of these women. Some of the more assertive tactics used in data collection for women who delivered a live birth would not be appropriate for women who have experienced a stillbirth. Sensitivity considerations should be built in to all components of the data collection methodology.
PRAMS grantees offer incentives in the form of gifts and/or rewards to women who are sampled to participate in the survey. In general, gifts and rewards have been found to be important for encouraging participation in federal surveys, especially for more reluctant responders. For PRAMS increasing motivation to participate through interventions and rewards is particularly salient: this survey focuses on a special population during a limited time period following the birth of an infant when women indicate that participating in even simple activities is constrained by lifestyle changes, financial constraints, childcare duties, and fatigue. Give these constraints, effective gifts and rewards have been found important for PRAMS and have worked to ensure that all populations of women, including minority women, and women with lower education or literacy levels, lower income, and less familiarity or trust in government surveys are sufficiently represented. Many grantees conduct experiments to determine what kind of gift or reward is most effective in motivating survey response
States must maintain training for all interviewers involved in PRAMS. Data collectors also participate in monthly or bimonthly conference calls with their CDC program manager to review operations and response rates. Issues related to response rates are discussed on these calls. In addition, a list server of all PRAMS data collectors is available for discussion and dissemination of various approaches to improve response rates.
Although response rates overall for surveys are declining, PRAMS maintains a relatively high level of response when compared to other mixed mode surveys. The AAPOR weighted response rates #6 for PRAMS states in 2016 ranged from 45% in Alabama to 73% in New York City, with a median rate of 60%. We are aware of no other U.S. population based health surveys using the same methodology as PRAMS. The table below provides some comparisons of the PRAMS response rate when compared to other, similar surveys.
Table B.3-1. Comparisons of Survey Response Rates
|
Response Rates |
|||
Survey |
Year(s) |
Overall |
Landline |
Cell Phone |
PRAMS1 |
2016 |
60% |
|
|
California Maternal Infant Health Assessment (MIHA)2 |
2012 |
70% |
|
|
National Immunization Survey (NIS)3 |
2011 |
|
61.7% a |
25.2% |
National Survey of Children’s Health (NSCH)4 |
2011-2012 |
|
54.1%b |
41.2% |
|
|
|
|
|
Behavioral Risk Factor Surveillance System (BRFSS)5 |
2016 |
|
47.7% |
46.4% |
National Health Interview Survey (NHIS)6 |
2016 |
54.3% |
|
|
National Health and Nutrition Examination Survey (NHANES)7 |
2015- 2016 |
58.7% |
|
|
1PRAMS response rate presented here is the median rate for all sites |
||||
2http://www.cdph.ca.gov/data/surveys/MIHA/Documents/MIHAOverviewWeb.pdf |
||||
3http://www.cdc.gov/vaccines/stats-surv/nis/dual-frame-sampling-08282012.htm; NIS does not include household sampling in the landline portion of the study but interviews the adult who self-identifies as "most knowledgeable" about household immunization information. |
||||
4 http://www.cdc.gov/nchs/slaits/nsch.htm; the NSCH interview completion rate defined as the proportion of households known to include children that completed all sections of the survey. |
||||
5 https://www.cdc.gov/brfss/; BRFSS response rates are presented here as median rates for all states and territories |
||||
6 https://www.cdc.gov/nchs/nhis/index.htm; (unconditional) final sample adult component response rate |
||||
7 https://www.cdc.gov/nchs/nhanes/index.htm; unweighted response rate |
||||
Response rates, cooperation rates, and refusal rates for PRAMS are calculated using standards set by the AAPOR [1]. PRAMS calculates response rates using AAPOR Response Rate #6, cooperation rates using AAPOR Cooperation Rate #2, and AAPOR Refusal Rate #3.
Based on the guidelines of AAPOR, response rate calculations include assumptions of no cases of unknown eligibility among potential respondents that are not interviewed. The eligibility requirements are implemented during the sampling process. The only situations where sampled women are ineligible include cases of adoption or surrogate birth that were not noted on the birth certificate or unusual circumstances where the eligibility requirements were not properly implemented during the sampling process. Since PRAMS utilizes a sampling design with unequal probabilities of selection, weighted response rates are calculated.
PRAMS protocols have been adapted over time to meet the needs of the data collection process and maximize response rates while minimizing respondent burden. PRAMS continually assesses its methods and procedures through comparisons with industry standards, consultation with PRAMS coordinators and other experts in the field, and real-world experience and feedback from the PRAMS data collectors. All questions which are included in the PRAMS are cognitively tested by CDC, except for state-added questions. Field tests are also conducted prior to any questionnaire revision to ensure it is ready for fielding. New or substantially revised questions are tested before adoption. The methods by which questions are adopted are provided in Attachment 3. As this document indicates, Subject Matter Experts from CDC and other federal agencies, state health department representatives and survey experts are involved in the process of question development. Many of the questions which are included in the PRAMS appear on other surveys including the National Health Interview Survey (NHIS), the National Adult Tobacco Survey (NATS), the National Health and Nutrition Examination Survey (NHANES), the National Immunization Survey (NIS) and others. The use of identical or similar questions is advantageous in that it allows researchers to make comparisons across different samples, different geographic areas or over time. PRAMS questions are cognitively tested by the Collaborating Center for Questionnaire Design and Evaluation Research hosted by NCHS, OMB No. 0920-0222, Exp. 08/31/2021. Additional field testing of new or substantially modified questions are conducted using PRAMS Field testing Methodology (Attachment 17).
CDC personnel are responsible for all statistical aspects of the PRAMS including data analyses and reporting. The following staff members are primarily responsible for PRAMS data reporting.
Name |
Title |
Phone |
|
Leslie Harrison |
Team lead, PRAMS |
770-488-6335 |
lfl0@cdc.gov |
Denise D’Angelo |
Project Officer |
770-488-6288 |
dnd1@cdc.gov |
Holly Shulman |
Statistician |
610-690-0861 |
hbs1@cdc.gov |
Aspy Taraporewalla |
IT Project Manager |
770-488-6222 |
est3@cdc.gov |
Brian Morrow |
Statistician |
770-488-6349 |
bxm0@cdc.gov |
Toyia Austin |
Data Manager |
770-488-5432 |
tfa0@cdc.gov |
The following external consultants have provided input on PRAMS methodology and the analysis of PRAMS data:
Name |
Affiliation |
Area of Consultation |
Don Dillman, PhD |
Washington State University |
Survey methodology |
Donna Brogan, PhD |
Emory University |
Analysis of complex survey data |
1. The
American Association for Public Opinion Research. 2015. Standard
Definitions: Final Dispositions of Case Codes and Outcome Rates for
Surveys. 8th
edition.
AAPOR.
http://www.aapor.org/AAPORKentico/AAPOR_Main/media/publications/Standard-Definitions2015_8theditionwithchanges_April2015_logo.pdf
2. The Council of American Survey Research Organizations. 2013. Code of standards and ethics for market, opinion, and social research http://c.ymcdn.com/sites/ www.casro.org/resource/resmgr/code/september_2013_revised_code.pdf?hhSearchTerms=%22casro+and+response+and+rate%22
3. Mail and Internet surveys: The tailored design method--2007 Update with new Internet, visual, and mixed-mode guide. Dillman DA. John Wiley & Sons. 2007.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | cww6 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |