Attachment G2 -
Report of Cognitive Testing of the 2012 Current Population Survey’s Fertility Supplement
Rachel Freidus, Theresa DeMaio and Katherine Drom
U. S. Census Bureau
Center for Survey Measurement
January 12, 2012
The Center for Survey Measurement conducted cognitive testing of the 2012 Current Population Survey’s Fertility Supplement. Revisions have been made to the questionnaire since its last administration in 2010.
Changes to the questionnaire include reorganization of the content to: 1) switch from asking about the birth of the woman’s last child to ask about the birth of her first child; 2) discontinue asking about the month of birth; and 3) ask follow-up questions about marital and cohabitation status at the time of the first birth.
In
November and December, 2011, 18 interviews were conducted in two
rounds. The first round consisted of nine interviews. After we made
necessary changes to the questionnaire, another nine interviews were
conducted. The interviews were conducted by the authors of this
report.
Respondents were recruited through advertisements
in local newspapers and Craigslist.com, through our respondent
database, and through personal networks. Respondents targeted were
women between the ages of 15 and 50. These women ranged from
having never given birth, to having given birth to eight children.
The table below documents the characteristics of the
cognitive interview respondents:
Age |
Number of Rs |
20-30 |
7 |
31-40 |
5 |
41-50 |
6 |
|
|
Race |
|
White |
4 |
Black |
10 |
Hispanic |
3 |
Mixed |
1 |
|
|
Education |
|
Some High School |
2 |
High School Graduate |
4 |
Some College |
7 |
College Graduate |
2 |
Advanced Degree |
3 |
During the interviews, respondents were instructed to “think aloud” as they were answering the survey questions. Pre-scripted probes were asked to determine their interpretation of specific terminology in the questions, and unscripted probes were asked whenever respondents seemed to be having difficulty responding to a question. Respondents were paid $40 for their participation in the interviews, which lasted 30 minutes on average. They were conducted either at the Census Bureau’s cognitive laboratory or at places convenient to respondents such as libraries and local commercial establishments.
A
copy of the questionnaire that we used in Round One is included as
Attachment A.
RESULTS
AND RECOMMENDATIONS
Results
are presented in two sections. First we present the results for new
and revised questions that were the main focus of the pretesting,
along with their associated recommendations. Following that, we
present findings related to one question from the background section
of the questionnaire. This question had a potential problem and we
present it for the sake of completeness.
Tested
Questions:
Within the write up for each question, we include the results of both rounds when changes were made between rounds. A copy of the questionnaire used for Round Two is included as Attachment B. This is also our recommendation for the final questionnaire, as we did not make any revisions after Round Two.
Supplement questions:
The following questions are about women and the number of children they have had. |
This
instruction was read to all 18 respondents during both rounds.
Respondents did not have any difficulty understanding it.
During
Round One, however, after the interviewer had read this instruction,
one respondent said “six,” telling the interviewer the
number of children she had, before any question had been asked.
Recommendation: We do not recommend any changes to this introduction.
SF1. How many live births, if any, have you ever had?
(Do
not include any stillbirths or adopted children)
If >0 go to SF2, If = 0 END.
|
This
question was asked of all respondents. During Round One, responses
ranged from none to six live births.
During Round One all
of the respondents answered this question correctly, though some were
not able to verbalize what “live births” meant to them
when we probed them. One respondent thought that a “live birth”
meant “giving birth to a human” and another could not
explain what the term meant. Others correctly believed that the term
“live births” was being used “to distinguish
between a real birth and a baby who may have died in utero,”
and to count births “where they actually survived.”
Respondents knew not to count abortions or miscarriages and
understood that “live births means children that survive the
birth and who are not stillborn” and “actually
survived.”
One respondent during Round One who had
given birth to twins was initially not sure if she should say that
she had one or two live births. When the respondent gave her final
response she counted the twins as two separate births, which was
correct.
This question, however, elicited an extremely
emotional reaction from one respondent who had a miscarriage. This
respondent felt the term “live births” made her think
about her miscarriage and thought we should have asked “how
many children do you have?” instead. This respondent said, “I
don’t like the way that was worded. It’s a touchy
subject…Women go through so many issues and they can be
emotional…My mind immediately went back to some unfavorable
situations.” The interviewer asked her why the term “live
births” made her think of those situations and she said,
“Because I went through a miscarriage and hearing ‘live
births’ makes me think I had a live birth gone wrong.”
This respondent voiced her opinion and thought that this question was
too private and too difficult for her to respond to. She felt that
the wording of the question took her “someplace mentally,”
and said that if she had been at home with her ex during an actual
interview she probably would have picked an argument with him because
he didn’t give her a “live birth.” The respondent
didn’t think the term “live births” was appropriate
for the Census Bureau to ask.
There were a few other
respondents who agreed that this question was worded in a sensitive
manner and that it elicited thoughts of stillbirths or miscarriages,
though they did not react as strongly as the aforementioned
respondent.
As a result of the level of sensitivity of
the question during Round One, the interviewers recommended that the
wording be changed. The sponsors agreed to change the wording to the
wording that is being used in the Survey of Income and Program
Participation. The question was revised to read:
Altogether how many children have you ever given birth to?
(Count all biological children of this person, regardless of whether they were born within or outside of any marriage. Do not count adopted, foster, or stepchildren; do not count stillbirths).
(Enter (N) for none)
Number:
____________
During Round Two, responses ranged from none to eight. All respondents answered this question correctly and none had difficulty understanding or verbalizing the meaning of this question. One respondent correctly pointed out that the question was asking “how many children did I actually have?” This respondent contrasted live births with pregnancies that did not result in live births. She was aware that miscarriages and abortions should not be counted.
Another respondent correctly excluded a miscarriage that she had. This respondent did not feel that the Round Two question wording was sensitive and did not feel that there was a different way to phrase this question that would have made her feel more comfortable. In fact, none of the respondents during this round felt that the question was sensitive. One respondent mentioned that maybe someone who had a stillbirth or miscarriage would find the question to be sensitive; however, this was not the case for the respondent who had a miscarriage.
Recommendation:
We do not recommend any further changes to this question.
SF2. In what year was your <child/first child> born?
|
This is a new question, which replaced a question about the birth of the last child. This question was asked seven times in Round One.
During Round One, most respondents did not have difficulty understanding or correctly answering this question, though one respondent hesitated to think about her answer before responding.
No
changes were made to this question between Rounds One and Two.
This
question was asked eight times in Round Two. During Round Two, most
respondents did not have difficulty understanding or correctly
answering this question, though one respondent said she had trouble
remembering the answer to this question because she has eight
children. This respondent thought and then answered that her oldest
child was born in 1979. However, she later mentioned that this child
is now 34 (which he would not be if he were born in 1979). This
respondent was either mistaken in the year of birth or in the age.
She admitted that she remembers her younger children’s
birthdays but not those of her older children.
Recommendation: We do not recommend any changes to this question.
SF3a. At
the time when your <child/first
child> was born were
you married? (If
married or separated mark “Yes” if widowed, divorced
or never married mark “No”) Yes END No
|
This question about the respondent’s marital status at the time of the first birth is new to the fertility supplement. This question was asked seven times in Round One.
During Round One, most respondents did not have difficulty understanding or correctly answering this question, though two respondents hesitated before answering the question. One said she had to think before answering because she was married when she had her second child but not when she had her first. Another said she had to think because she was married when her first child was born but not when this child was conceived.
No changes were made to this question between Rounds One and Two.
This question was asked eight times in Round Two. During Round Two, respondents did not have any difficulty understanding or correctly answering this question.
Respondents
did not perceive this question to be sensitive.
Recommendation: We do not recommend any changes to this question.
SF3b. At the time when your <child/first child> child was born, did you have a boyfriend or partner in the household? (If SF3A = No)
Yes No
|
This question about the respondent’s cohabitation status at the time of the first birth is new to the fertility supplement. This question was asked four times in Round One.
During Round One, respondents did not have difficulty understanding or correctly answering this question. Through probing, interviewers learned that the respondents who said yes correctly reported that they lived with their significant others.
No changes were made to this question between Rounds One and Two.
This question was asked five times in Round Two. Most respondents did not have difficulty understanding or correctly answering this question, though two respondents hesitated before answering the question. One respondent hesitated, not because she didn’t understand the question, but because she said she wasn’t sure how to answer the question. She said that due to circumstances, she was not sure whether her son’s father was her boyfriend or not at the time of the birth. At the end of the interview, this respondent said she thought this question was sensitive, as she had a negative relationship with her son’s father. Another respondent hesitated because she had to think about whether she was or was not in a relationship with her boyfriend at the time of the birth.
Aside from the one respondent, the other respondents did not perceive this question to be sensitive.
Recommendation: We do not recommend any changes to this question.
Background Questions:
One question in the initial section of the questionnaire presented an interesting situation and we present it in the interest of completeness.
|
This question was asked a total of 10 times, five times in the first round. In the first round respondents answered “Yes” twice, “No” twice, and one respondent answered “boyfriend” thinking it was a multiple choice question instead of a yes or no question. One respondent who answered “Yes” commented that this question was confusing because she is currently living with an ex-boyfriend. This may be a situation that occurs with some frequency given the current state of the economy.
In the second round of testing this question was asked four times, and respondents answered “No” three times and “Yes” only once. Respondents did not display any difficulty with answering this question in the second round of testing.
There were no changes made to this question between the first and second round of testing.
Recommendation: We do not recommend any changes to this question.
Attachment A- 2012 Current Population Survey’s Fertility Supplement Initial Questionnaire
Background questions:
|
|
|
|
|
|
2 1st, 2nd, 3rd or 4th grade 3 5th or 6th grade 4 7th or 8th grade 5 9th grade 6 10th grade 7 11th grade 8 12th grade NO DIPLOMA 9 HIGH SCHOOL GRADUATE- high school DIPLOMA or the equivalent (For example: GED) 10 Some college but no degree 11 Associate degree in college - Occupational/vocational program 12 Associate degree in college -- Academic program 13 Bachelor's degree (For example: BA, AB, BS) 14 Master's degree (For example: MA, MS, MEng, MEd, MSW, MBA) 15 Professional School Degree (For example: MD,DDS,DVM,LLB,JD) 16 Doctorate degree (For example: PhD, EdD)
|
|
|
|
.
that
you consider yourself to be: White; Black or African American;
American Indian or Alaska Native; Asian; OR Native Hawaiian or
Other Pacific Islander.
2 Black or African American (Skip to Q15- SF1) 3 American Indian or Alaska Native (Skip to Q15- SF1) 4 Asian (Skip to Q12) 5 Native Hawaiian or Other Pacific Islander (Skip to Q13) 6 Other - DO NOT READ (Skip to Q14) |
2 Chinese 3 Filipino 4
Japanese 6 Vietnamese 7 Other Asian - DO NOT READ. Specify ______ |
1 Native Hawaiian 2 Guamanian or Chamorro 3
Samoan |
|
Supplement questions:
The following questions are about women and the number of children they have had. |
(Do
not include any stillbirths or adopted children)
If >0 go to SF2, If = 0 END.
|
If > 1 read <first> for the remainder of the questions. |
|
(If
married or separated mark “Yes” if widowed, divorced
or never married mark “No”) Yes END No
|
Yes No
|
Attachment B- 2012 Current Population Survey’s Fertility Supplement Round Two and Recommended Questionnaire
Background questions:
|
|
|
|
|
|
2 1st, 2nd, 3rd or 4th grade 3 5th or 6th grade 4 7th or 8th grade 5 9th grade 6 10th grade 7 11th grade 8 12th grade NO DIPLOMA 9 HIGH SCHOOL GRADUATE- high school DIPLOMA or the equivalent (For example: GED) 10 Some college but no degree 11 Associate degree in college - Occupational/vocational program 12 Associate degree in college -- Academic program 13 Bachelor's degree (For example: BA, AB, BS) 14 Master's degree (For example: MA, MS, MEng, MEd, MSW, MBA) 15 Professional School Degree (For example: MD,DDS,DVM,LLB,JD) 16 Doctorate degree (For example: PhD, EdD)
|
|
|
|
that
you consider yourself to be: White; Black or African American;
American Indian or Alaska Native; Asian; OR Native Hawaiian or
Other Pacific Islander.
2 Black or African American (Skip to Q15- SF1) 3 American Indian or Alaska Native (Skip to Q15- SF1) 4 Asian (Skip to Q12) 5 Native Hawaiian or Other Pacific Islander (Skip to Q13) 6 Other - DO NOT READ (Skip to Q14) |
2 Chinese 3 Filipino 4
Japanese 6 Vietnamese 7 Other Asian - DO NOT READ. Specify ______ |
1 Native Hawaiian 2 Guamanian or Chamorro 3
Samoan |
|
Supplement questions:
The following questions are about women and the number of children they have had. |
(Count all biological children of this person, regardless of weather they were born within or outside of any marriage. Do not count adopted, foster, or stepchildren; do not count stillbirths).
(Enter (N) for none)
Number: ____________
If
>0 go to SF2, If = 0, END.
|
If > 1 read <first> for the remainder of the questions. |
|
(If
married or separated mark “Yes” if widowed, divorced
or never married mark “No”) Yes END No
|
Yes No
|
| File Type | application/msword |
| Author | freid002 |
| Last Modified By | woods009 |
| File Modified | 2012-01-30 |
| File Created | 2012-01-30 |