Childhood & Family Experiences
New Collection
Supporting Statement
Part B
February 2019
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officer:
Amelia Popham
This document discusses the planned statistical methods for the Childhood and Family Experiences (C&FE) study. This study is sponsored by the Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS). This submission seeks OMB approval for four data collection instruments that will be used as part of the sample selection and interview process:
Phone Screener for Prospective Families (Appendix A)
Adult Interview Guide (Appendix B)
Adolescent Interview Guide (Ages 12-17) (Appendix C)
Child Interview Guide (Ages 7-11) (Appendix D)
This justification provides supporting statements for each of the five points outlined in Part B of the OMB guidelines.
The respondent universe for this study includes children and families living in poverty. The research team will interview 30 families in three cities or communities and will work with organizations in the selected areas to recommend families for participation. Specifically, the research team will interview: 1) parents or parent figures of children living in poverty; 2) adolescents living in poverty; and 3) children living in poverty. Adolescents are defined as those ages 12-17 and children are defined as those ages 7-11.
The research team will take two main considerations into account when selecting cities or communities as study sites. Site selection decisions place approximately equal weights on these considerations:
Representativeness: The research team seeks to reflect a range of experiences of parents and children in poverty in the U.S. Thus, site selection will consider policy, programmatic, and geographic characteristics along a number of dimensions that may influence those experiences. Policies and benefit levels vary considerably across states for Temporary Assistance for Needy Families (TANF)1 and for Medicaid and the Children’s Health Insurance Program (CHIP).2 While the availability of social service supports might be greater in more densely populated areas, accessibility of social services to those who need them may be problematic in both urban and rural areas.3 A key question is whether perceptions of those in poverty differ by geographic region or metro/non-metro area. The research literature does not address this issue directly and the research team seeks to help fill that gap. Some studies suggest that children first become aware of social status through comparisons of their own goods and lifestyle markers (e.g., where vacations are spent) with those of others.4 By extension, children in more rural areas that are fairly income-homogeneous may develop this awareness relatively later than children in urban areas.5 Still, children may develop comparisons with others through the media or online, and thus may develop comparisons with others regardless of their location. The research team is also attuned to associations between geographic location and sample characteristics that are not driven by policy – for example, race, ethnicity, and language.
Logistics and costs: The research team expects to work with local social service agencies to identify and reach out to possible sample participants. It will be much more effective and realistic to work with organizations where the research team or consultants have existing relationships from working together in previous research projects or through other professional networks. Local organizations may be more open to serving as intermediaries and referral sources for potential study participants if they have built trust with the research team through previous working relationships. We will also prioritize some sites that are closer to research team members as a secondary cost consideration. Study sites that are nearer to our research staff, consultants, or other project contacts are more practical and cost-effective for conducting outreach and completing interviews for sample members within budget.
The process of engaging sites and establishing local contacts for the study will consist of several steps.
Introduction to the Project: Once the research team has identified potential sites, they will make initial phone calls to have individualized conversations with key contacts in those sites. Key contacts are program managers or department directors with whom the team has existing relationships through previous research projects or other professional networks. Research team members will provide a preliminary introduction to the study and describe the sample eligibility criteria to determine the contact’s ability to help identify potential eligible study participants. Upon completion of this stage the research team will finalize a list of up to three sites and determine who the primary contact will be for the next step.
Preliminary Planning: After the initial phone call with potential organizational contacts, the research team will schedule a follow-up individualized conversation to discuss in greater detail the process to identify or refer potential families to the project. Organizational staff may choose to reach out to study participants directly, or they may provide names and contact information of potential participants to the research team. If the local organization wants to be involved in engaging families we will provide them with a brief Staff Outreach Script (Appendix J) to help facilitate this process. Once contact information for potential families is obtained, research team members will use a brief screener to assess eligibility via phone (Appendix A). This call will be used to assess families’ eligibility for the study and, for those who are eligible, provide them with additional information about the study, including any risks, and to assess their interest in participating.
A brief phone screener for prospective families (Appendix A) will be used to ascertain families’ eligibility for and interest in participating in the study. The research team will consider the following criteria when selecting families to participate in the study:
Characteristics of the families: The research team is interested in the following characteristics:
Household or family income: The study seeks to identify families living in deep poverty (less than 50 percent of the federal poverty level). As discussed in Section B.1.2, the study team will work with local organizations to identify families who are likely to meet this criterion (for example, through their knowledge about a family’s benefit receipt). To the extent possible, the sample would reflect diverse types of benefit receipt (TANF, Medicaid, CHIP, SNAP, housing assistance, WIC, free and reduced-price school lunch program, EITC) as well as no benefits.
Race and ethnicity: The study seeks to reflect diversity in its sample of families. Site selection will play a role in the pool of families likely to be recruited within each site.
Language spoken at home: Among people ages five and older who are in poverty, almost 30 percent speak a language other than English at home.6 Just under half speak English “very well.” Including some sample members whose primary language is Spanish might be most effectively and efficiently met by selecting an urban site with a large Hispanic population.7
Number of children in the family: The research team anticipates interviewing one to two children per family. The research team will keep track of the number of children in each age group interviewed at each site to ensure that the study is conducting approximately seven adolescent interviews and ten child interviews per site. When there are more than two eligible children in a family, the research team will use the following decision rules:
Interview at least one child in the 7-11 age group (select the youngest child);
If another child is in the 7-11 age group, interview that child (select next-to-youngest); otherwise interview the adolescent in the 12-17 age range (if multiple adolescents in this age range, select the youngest).
Characteristics of the adults: The research team is interested in adults’ relationships to children in their family, their relationship status, and their work status. Most adults in the sample are likely to be biological (or adoptive) parents of the children in their household. Other parent figures such as grandmothers or aunts who are primary caregivers for a child are eligible to be included in the sample as well. Both one- and two- parent families will be eligible for the sample. In two-parent families, we will interview either both parents together, or one parent (but not each parent separately). We will follow parents’ preferences. Because families in poverty who have children younger than 18 are more likely to be headed by females,, we anticipate a higher percentage of female respondents.8 The study team anticipates the sample will include employed, under-employed, and unemployed adult respondents.
Characteristics of the children: The research team will select families who have:
Children ages 7 to 17 (with at least half ages 7 to 11 years, and no more than half ages 12 to 17 years)
Once the research team has finalized the list of sites, they will begin conversations with local organizations in those cities to begin identifying families for participation. Based on field experience in previous qualitative studies, the research team estimates needing to start with a contact list of 30 to 40 families in a site, in order to recruit and complete interviews with 10 families. The research team will draw on relationships with partner agencies in these sites such as local TANF agencies, SNAP offices, community based organizations, or schools to recruit study participants. Organizational staff may choose to reach out to study participants directly, or they may provide names and contact information of potential participants to the research team. If and when the local organization wants to be involved in engaging families we will provide them with a brief Staff Outreach Script (Appendix J) to help facilitate this process. Once contact information for potential families is obtained, research team members will use a brief screener to assess eligibility via phone (Appendix A). This call will be used to assess families’ eligibility for the study and, for those who are eligible, provide them with additional information about the study, including any risks, and to assess their interest in participating.
The research team will schedule interviews with families in each site that are interested and able to participate. The sample goal is 10 families per site. Interviews will most likely be scheduled in the families’ homes. Some families may prefer meeting in alternative locations in the community, such as a church, community center, restaurant, public library, community-based organization, or public park. Trained staff will use interview guides included in Appendices B-D to conduct hour and a half-long interview for adults, and based on pre-testing of the interviews 45-60 and 30-minute-long interviews for adolescents and children, respectively (see Section B.4).
Each set of family interviews will include a separate interview with one to two children ages 7 to 17 years old in the family; and a separate interview with one to two parents or parent figures. Interviews will be conducted around roughly the same time period for each of the sites to minimize the chances that any random, unanticipated events (e.g., a natural disaster or some other environmental condition such as inclement weather) or holidays do not systematically influence either the manner in which the data were collected or the response that the interviewees provide. Some variation in timing is expected, given the time it takes to recruit families, schedule interviews, and complete interviews for a family, and taking into account field staff availability and proximity to the site.
Interviewers within and across the sites, along with project supervisory staff, will meet regularly once data collection is underway to review the interview procedures and experiences. In-person interviews allow for the collection of non-verbal information (e.g., general attitude of the interviewee, assessments of rapport built during the interview, interviewees’ interactions with their surroundings) that other data collection methods do not. Interviewers will take detailed notes during each interview, capturing both verbal and non-verbal information. All interviews will be audio recorded (if consent/assent are provided) and the recordings will be transcribed. Interviewers will write summary notes after each interview to record early impressions and themes that emerged during the interview. These notes will not be considered project data to be analyzed but they will serve as a record of early impressions and initial (emerging) themes that will then be followed up on by a more thorough and systematic analysis of the interview data.
To gain an understanding of adults’, children’s, and adolescents’ experiences of poverty, the data will be coded in an iterative process, using software specifically designed for working with qualitative interview data (for example, Dedoose).9 In coding of the data and identifying themes and categories, the research team will pay particular attention to comparisons among households that have different compositions (e.g.,, households with younger/older children, single parent homes/two parent homes/extended families), and contextual circumstances (for example, geographic location) that might shape the lives of households living in poverty and their understanding of their circumstances.
Because the research team will have conducted a screener outreach to assess families’ willingness to participate prior to enrolling them in the study, and are proposing varying incentives for adults, adolescents, and children, we expect to achieve the sample goal of around 45 adults and 50 children.
We propose providing each interviewed adult with a $40 gift card, and each interviewed adolescent with a $25 gift card. We propose that children (ages 7-11) choose items from a “treasure box,” valued at up to approximately $10. These incentives are proposed in order to reduce nonresponse bias from potential sample members in the deepest poverty and with the lowest education levels as discussed in Supporting Statement A. Respondents have the option of skipping an interview question; however, in order to reduce item nonresponse bias, topics can be revisited during the interview. As interviewers build rapport with respondents and they become more comfortable with the process, interviewers have the option of reframing questions or using probes in order to learn more information about a topic. Additionally, interviewers will gauge possible reasons for interviewee nonresponse and respond accordingly. For instance, if children are tired or bored, interviewers will offer breaks or opportunities to get up a stretch to help children stay focused. If the interviewee seems uncomfortable with the content of the questions, interviewers will reassure the adults that their names will not be used in any reports and that their responses will not affect their current and future benefits and remind children and adolescents that there will be no ramifications within their family.
The methodology of data collection is designed to maximize response rates. For instance, the interview guides were developed with expert feedback and field tested with fewer than 10 people in order to ensure all questions are developmentally age-appropriate and easy for all participants to understand. In addition, interviewers will be trained to make respondents comfortable with the interview process and learn how to probe appropriately when needed. Additionally, participants will know up front what to expect from the interview before they are enrolled in the study. Additionally, offering incentives to potential sample members can maximize response rates. Lastly, interviews will be scheduled at both a time and place convenient for participants, and interviewers will explain participants’ right to privacy upfront before beginning the interview.
Pretesting and piloting of nine or fewer child or adolescent interviews will involve:
A period of pre-testing with three children and two adolescents to inform modifications to the interview protocols.
One round of pilot testing in the field at each of the three sites that will simultaneously be used for training purposes and to finalize the protocols
Pretesting: Pretesting activities have been completed and included practice administration of the interview protocols conducted with three children and two adolescents of similar ages and backgrounds to the study participants. Four of the five interviews occurred within the expected amount of time. One child interview lasted approximately 45 minutes. To ensure that interviews last the planned amount of time, the research team will focus on training interviewers to manage the flow of the conversation and find a balance between focusing on core questions and probing for additional in-depth information.
Piloting: Once the child interviews are deemed to be near final in content, structure, and length, we will conduct a final round of pilot testing (one interview per interviewer) that will simultaneously serve to help train and certify individuals who will be fielding the child and adolescent interviews and to finalize the content of the interviewer procedures manual.
The number of individuals participating in pretesting and piloting prior to the study receiving OMB approval will not exceed nine per Paperwork Reduction Act guidelines. If necessary changes are identified to interviews, updated materials will be provided to OMB for review and approval prior to use in the field. We will discuss with OMB/OIRA if these changes can be made through a nonsubstantive change request.
Exhibit B-1 lists individuals who contributed to the design of the interview guides and/or will be responsible for collecting and analyzing data.
Exhibit B-1: Individuals Involved in Instrument Development and Data Collection/Analysis
Individual |
Affiliation |
Role in Study |
Susan Clampet-Lundquist |
St. Joseph’s University |
Consultant |
Kathy Grant |
Depaul Universtiy |
Consultant |
Katherine Griffin |
UCLA |
Consultant |
Linda Mayes |
Yale University |
Consultant |
Rashmita Mistry |
UCLA |
Consultant, interview, and data analysis |
Carolyn Hill |
MDRC |
Data analysis |
Linda Mayes |
Yale |
Interview and data analysis |
Victoria Quiroz-Becerra |
MDRC |
Interview and data analysis |
Samantha Wulfsohn |
MDRC |
Interview and data analysis |
Annie Utterback |
MDRC |
Interview and data analysis |
Rachael Metz |
MDRC |
Interview and data analysis |
Doug Phillips |
MDRC |
Interview and data analysis |
Marissa Strassberger |
MDRC |
Interview and data analysis |
Toni Castro-Cosio |
MDRC |
Interview and data analysis |
Ann Bickerton |
MDRC |
Interview and data analysis |
Osvaldo Avila |
MDRC |
Interview and data analysis |
Emily Partin |
Sewanee: The University of the South |
Interview |
Amelia Popham |
Office of Planning, Research, and Evaluation |
Project Officer |
Lauren Deutsch |
Office of Planning, Research, and Evaluation/Business Strategies Consultants |
Contract Project Officer |
1 Urban Institute. (2016). “Welfare Rules Database.” Website: www.wrd.urban.og; U.S. Department of Health and Human Services. (2016) Welfare Rules Databook: State TANF Policies as of July 2015. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
2 Medicaid. (2017). “Financing and Reimbursement.” Website: www.medicaid.gov/medicaid/financing-and-reimbursement
3 Allard, Scott W. (2009). “Mismatches and Unmet Needs: Access to Social Services in Urban and Rural America.” Pgs 337-368 in James P. Ziliak (ed.) Welfare Reform and its Long-Term Consequences for America’s Poor. Cambridge, England, UK: Cambridge University Press; Allard, Scott W. (2008). Out of Reach: Place, Poverty, and the New American Welfare State. New Haven, CT: Yale University Press.
4 Personal interview with Rashmita Mistry (January 5, 2017); Mistry, Spears Brown, Chow, and Collins (2012); Spyrou (2013).
5 Personal interview with Linda Mayes (December 20, 2016).
6 Ryan (2013), see Table 3.
7 Among the general population Spanish is spoken among 62% of those speaking a language other than English, followed by 4.8% speaking Chinese. The percentages for other languages range from 2.6%-.2% of those speaking a language other than English. Therefore, eligibility criteria to participate in the study are either English or Spanish Speaking. (Ryan, 2013).
8 U.S. Census Bureau, Current Population Survey, 2018 Annual Social and Economic Supplement. Table POV04: Families by Age of Householder, Number of Children, and Family Structure: 2017. Retrieved from https://www.census.gov/data/tables/time-series/demo/income-poverty/cps-pov/pov-04.html#par_textimage_14, February 8, 2019.
9 Dedoose Version 7.0.23, web application for managing, analyzing, and presenting qualitative and mixed method research data (2016). Los Angeles, CA: SocioCultural Research Consultants, LLC www.dedoose.com.
Supporting
Statements for OMB Clearance Request Part B
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-20 |