Supporting Statement A - AMCS Study - Formative generic

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Assessing Models of Coordinated Services for Low-Income Children and Their Families (AMCS)

OMB Generic Information Collection Request: Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part A

JULY 2019

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Laura Hoard and Ivelisse Martinez Beck

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a Generic Information Collection under the umbrella generic: Formative Data Collections for ACF Research (OMB #0970-0356).

• Description of Request: This generic information collection is proposed to verify ACF’s understanding of programs compiled during a national scan of publicly available information and to conduct telephone interviews with staff from a subset of state and local-level coordinated services approaches identified through the national scan. Specifically, we will be reaching out to staff who are engaged in approaches to coordinate early care and education with services aimed at family economic security and/or other health and human services at the state or local levels. These approaches may, for example, be housed in a state agency overseeing other early care and education services (e.g. Child Care and Development Fund, Head Start/Early Head Start). We will ask respondents about coordinated services approaches operating at the state or local level to verify the accuracy of ACF’s information and complete any information not available publically. Activities will inform:

  • selection of sites to visit for more in-depth fieldwork covered under a forthcoming full information collection request;

  • ACF leadership on the characteristics of state and local coordinated services approaches that are currently operating in the United States; and

  • technical assistance for Preschool Development Grant Birth to Five grantees.

We do not intend for the data we collect in the study to be generalized to a broader population.

A1. Necessity for Collection

There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose

Purpose and Use

The purpose of the Assessing Models of Coordinated Services for Low-Income Children and Their Families (AMCS) project is to understand how states and local communities are coordinating services across sectors to most efficiently and effectively serve low-income children and their families. Policymakers and program leaders across the country are experimenting with innovative approaches to combine early care and education, family economic security, and other health and human services (Hulsey et al. 2015; Ross 2018; Sama-Miller and Baumgartner 2017). These approaches vary along a range of dimensions, including their number and types of partners, funding streams, target populations, goals and objectives, locations, services provided, and monitoring processes. This qualitative study aims to fill gaps in our knowledge by identifying and describing the features of state and local approaches to coordinating early care and education services with family economic security and/or other health and human services.

The activities outlined in this generic information collection (GenIC) request will be used to inform the selection of six (6) coordinated services approaches for site visits (to be submitted through a future full information collection request (ICR)). Additionally, information from these activities will be used to inform ACF leadership about the characteristics of state and local coordinated services approaches that currently operate in the United States and provide technical assistance for Preschool Development Grant Birth to Five (PDG B-5) grantees. The PDG B-5 legislation states that the purpose of the grant is to “assist States to develop, update, or implement a strategic plan that facilitates collaboration and coordination among existing programs of early childhood care and education in a mixed delivery system across the State designed to prepare low-income and disadvantaged children to enter kindergarten and to improve transitions from such system into the local educational agency or elementary school that enrolls such children” (Section 9212 of the “Every Student Succeeds Act (ESSA), PL 114-95”). As States develop, update and implement coordinated services approaches in line with the PDG B-5 grant, it is imperative that they have information about best practices and lessons learned from existing coordinated services approaches. This study will provide information on existing coordinated services approaches that can be used to inform the work of the PDG B-5 grantees.

Research Questions or Tests

The study has six primary research questions (Table 1). These questions cover the full range of data collection activities of the study; the highlighted column shows the activities included in this formative data collection request. The information collected as part of this formative request will inform the data collection that is part of the future full information collection request.

Table 1. Research questions and sources of information

* Note: the size of the check mark denotes how intensely the data collection methods will be able to answer the research question.

Research questions	Current GenIC	Future ICR
	Model profile data Staffing information Telephone interviews	Site visits
1. Are there coordinated services approaches that “work,” such that partnerships and coordination of service application and delivery are coordinated? Can we identify key characteristics that make the approaches “work”?		
2. How do coordinated services approaches intend to reduce barriers and road blocks for families to access services? Are there federal barriers to implementing such approaches?		
3. Are approaches that combine ECE, family economic security, and/or other health and human services able to address other child development factors beyond ECE?		
4. What have we learned from efforts to integrate enrollment and eligibility processes for health and human services?		
5. Are states and/or localities examining service delivery dynamics across ECE programs to assess availability of care slots and services to meet the needs of eligible families? How are they using data to understand service delivery dynamics?		
6. How is public and private ECE funding targeted to meet the needs of at-risk children and families? Are there differences in the families that are able to access services?		

ECE= early care and education.

Study Design

The AMCS study has a descriptive, qualitative design.

To build knowledge from publicly-available information and reduce burden on potential respondents, ACF conducted a national scan of state and local coordinated service approaches. This scan included a search of publicly available information (a non-burden activity) to identify and summarize the characteristics of state and local coordinated services approaches that are currently operating in the United States. This information will be used to create state and local model profiles. Criteria for inclusion in the scan can be found in section B2. Data collection activities for this GenIC will focus on descriptively addressing research questions 2, 4, 5, and 6 listed in Table 1.

Our proposed descriptive data collection has limits. The information gathered through the national scan and used to populate the state and local model profiles relied on information in the public realm at the time the national scan was conducted, and thus the proposed ICR may exclude some current coordinated services approaches and details that had not been documented in the publically available sources of information. Further, this ICR can only capture a descriptive picture of what coordinated services approaches are doing and what they have learned about the work; we cannot measure or evaluate the effectiveness of the coordinated services approaches. We will take care in the presentation of findings to ensure they are interpreted as descriptive in nature and that they do not speak to the effectiveness of approaches. The findings of this ICR cannot be generalized beyond the approaches that were found through the national scan and that respond to requests for participation in the data collection for this project. Despite these limitations, collecting in-depth qualitative information about a set of approaches will provide important information to inform technical assistance efforts and future field work. As interest in coordinated services approaches grows, it is important to gather information about how approaches work and to understand staff experiences in coordinating service delivery. This information can serve as important context and foundation for future data collection efforts, which might include implementation, evaluation, or impact studies.

Table 2 provides information about each of the proposed activities for this study. These activities build on one another: the state and local model profiles provide a framework for thinking about the ways in which states and localities coordinate services, and we will use that information to select state and local approaches to participate in telephone interviews. Using model profile data and model staffing information as the basis for those telephone interviews, we can use the interviews as an opportunity to probe for in-depth information about partnership processes, model features, barriers to coordination, and other key aspects of the state and local coordinated services approaches.

We will use the formative information collected from these activities to inform our selection of six sites that we will visit to collect further qualitative information. These site visits, which will include semi-structured staff interviews and focus groups with parents, will be covered in a future full information collection request.

Table 2. Study design components and timeline

Study activity	Time frame	Respondent (s)	Content and Purpose	Mode and Duration
Model scan	October 2018–June 2019	N/A	Content: Broad search of publicly available information on approaches that coordinate ECE, family economic security and/or other health and human services. Purpose: Describe the general landscape of coordinated services approaches currently operating across the country.	N/A
Model profile data	August-September 2019 (Current GenIC)	One staff member from individual state and local coordinated services approach	Content: Model profile information. Purpose: Confirm and complete state and local model profile data.	Mode: Email and follow-up calls Duration: 2 hours
Staffing information	September-October 2019 (Current GenIC)	One staff member from individual state and local coordinated services approach	Content: Information about staff respondents for the telephone interview and organizational structure. Purpose: Provide contextual information for telephone interviews	Mode: Email and follow-up call Duration: 30 minutes
Telephone interviews	September-November 2019 (Current GenIC)	Staff from 20 individual state and local coordinated services approaches	Content: Information that extends model profile data across topics, such as development of the approach, coordination with partners, data systems and use, and services provided. Purpose: Understand features of coordinated services approaches and inform technical assistance and selection of sites to visit.	Mode: Interviews Duration: 1 hour
Site visits	Future ICR, expected Winter 2020	Staff from six individual state and local coordinated services approaches	Content: Detailed information about day-to-day operation of approach, including discussions with staff and families receiving coordinated services. Purpose: Develop an in-depth understanding of implementation.	Mode: Interviews and focus groups Duration: 1 day

The data collection procedures for study activities are included in Supporting Statement B.

A3. Use of Information Technology to Reduce Burden

ACF began by developing model profiles based on publicly available information found on the Internet and in publicly available reports. Under this ICR, the project team will reach out to staff from state and local approaches by email (following up to a subset of approaches by phone if necessary) to ask for confirmation and completion (if necessary) of the information included in the model profile summaries. In the telephone interviews, with permission from respondents, study staff will use an audio recording to capture details from the interviews, including direct quotes. The approach has been designed to reduce burden on state and local staff.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

The study will not collect information that is available from alternative sources. At each stage of the study, the study team will review notes from current activities to ensure that they do not duplicate information that can be reliably obtained through other sources, such as directly from the state or local approach website or from a verifiable source (e.g., a foundation report). For example, prior to conducting telephone interviews, the study team will review the model profile information and only ask questions in the telephone interview that have not previously been answered as part of the model scan. During profile review, state and local staff will be asked to fill in information only when no information was found online or to correct information that is inaccurate.

A5. Impact on Small Businesses

Some of the agencies included in the study will be small organizations, including community-based organizations and other nonprofits. The study team will minimize burden for respondents in the telephone interview by ensuring model profiles are as complete as possible based on available information and by inviting state and local approaches to confirm and complete model profile information via email. In addition, the study team will be sure to conduct interviews at times that are convenient for the respondents.

A6. Consequences of Less Frequent Collection

The requests for confirmation and completion of model profile data, requests for model staffing information, and telephone interviews are a one-time data collection activity. Collecting these data are necessary for ACF to gain a better understanding of how states and local communities coordinate services to serve families, how they coordinate funding and service delivery, and the barriers to and facilitators of these functions.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on October 11, 2017, Volume 82, Number 195, page 47212, and provided a sixty-day period for public comment. During the notice and comment period, no substantive comments were received.

Consultation with Experts Outside of the Study

Throughout the study, we have consulted with stakeholders and experts in the field. The stakeholders and experts represent the interests of the study at various levels, including federal, state, and local policymakers and program operators. Experts provided input on project products, including model profile templates and telephone interview protocols, and will offer information about existing models that might be appropriate for data collection. To date, the study team has consulted with experts within ACF, including leadership and staff from the Office of Child Care, who have helped to shape and advise on the study activities. We have also consulted with five coordinated services experts: Betina Jean-Louis (Harlem Children’s Zone); Anne Mosle (The Aspen Institute); Teresa Eckrich Sommer (Northwestern University); Peter Tatian (Urban Institute) and Kristin Bernhard (Ounce of Prevention).

A9. Tokens of Appreciation

We do not propose any tokens of appreciation for respondents for this information collection.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

This information collection request includes the collection of minimal personally identifiable information. Instrument 3, Staffing Information, includes a request for the names of staff members who will participate in the telephone interview. In order to build rapport for the interview, we need to be able to address staff by name, particularly in the context of a group interview. Not only is it respectful to refer to staff by name, but it will also help to make the interview efficient by allowing the interviewer to direct specific questions to the relevant staff. Staff names will not be connected with interview responses. Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, the Contractor, Mathematica, will comply with all Federal and Departmental regulations for private information.

Data Security and Monitoring

The Contractor developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ personally identifiable information. The Contractor shall ensure that all of its employees, who perform work under this contract, are trained on data privacy issues and comply with the requirements outlined in the Data Safety and Monitoring Plan. All of the Contractor’s staff sign the Contractor’s confidentiality agreement when they are hired.

As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system; establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor must submit a Data Safety and Monitoring plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access.

A11. Sensitive Information ¹

There are no sensitive questions in this data collection. Respondents can refuse to answer any question they do not wish to answer.

A12. Burden

Explanation of Burden Estimates

The current request includes burden estimates to cover the following activities:

Instruments 1 & 2: We will email the request to confirm and complete state and local model profiles to seventy-six (76) state and local coordinated services approaches. We estimate that it will take one respondent up to two hours to confirm the model profile data. This activity will only occur once over the request period. We anticipate that some state and local coordinated services approaches may require further follow up to confirm and complete the model profile data. We plan to follow up with a telephone call and estimate that we will speak to one respondent per approach for up to thirty (30) minutes to confirm and complete select questions from the model profile to prepare for the telephone interview. This activity will only occur once over the request period.

Instrument 3: We will request staffing information (organizational chart, names of staff who will join the telephone interview) from twenty (20) state and local coordinated services approaches that agree to participate in a telephone interview (Instrument 4). We estimate it will take one respondent thirty (30) minutes to gather an organizational chart, if available, and complete the staffing information table . This activity will only occur once over the request period.

Instruments 4 & 5: State and local telephone interviews will be conducted as small-group interviews with staff who were recommended through the request for staffing information... We estimate twenty (20) one-hour telephone interviews, with five respondents per interview. This activity will only occur once over the request period.

Burden estimates below include time for respondents to review instructions, search data sources, complete and review the responses, and transmit or disclose information. We expect the total annual burden to be 277 hours.

Estimated Annualized Cost to Respondents

We expect the total annual cost for respondents to be $8,531.00 for the information collection in the current request. Average hourly wage estimates for deriving total annual costs are based on Current Population Survey data for the third quarter of 2018 (Bureau of Labor Statistics 2018). For each instrument included in the burden table, we calculated the total annual cost by multiplying the annual burden hours and the average hourly wage. For respondents, we used the median usual weekly earnings for full-time wage and salary workers age 25 and older with a bachelor’s degree ($30.80 per hour). We divided weekly earnings by 40 hours to calculate hourly wages.

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Instrument 1-2: Confirmation of State or Local Model Profile Template	76	1	2	152	$30.80	$4,681.60
Instrument 1-2: Confirmation of State or Local Model Profile Template - Telephone follow-up	30	1	.5	15	$30.80	$462.00
Instrument 3: Request for Program Staffing Information	20	1	.5	10	$30.80	$308.00
Instrument 4-5: State or Local Telephone Interview Protocol	100	1	1	100	$30.80	$3,080.00
Estimated Annual Burden Total				277		$8,531.60

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

The total cost for the data collection activities under this current request will be $85,463. This amount includes all costs related to study design, development, field work, analysis and dissemination.

Cost Category	Estimated Costs
Instrument Development and OMB Clearance	$30,449
Field Work	$33,048
Publications/Dissemination	$21,966
Total costs over the request period	$85,463
Annual costs	$85,463

A15. Reasons for changes in burden

This is a new information collection request.

A16. Timeline

The study team will invite state and local coordinated services approaches to confirm and complete model profiles over a two month period following OMB approval. Requests for model staffing information and telephone interviews will take place following the confirmation and completion of model profiles and continue until completed, for about two months.

Information collected may be incorporated into documents or presentations that are made public. For example, research design documents or reports; contextualization of research findings from a follow-up data collection that has full PRA approval; or informational reports to TA providers. In sharing findings, we will describe the study methods and limitations with regard to generalizability and as a basis for policy.

The tentative timeline for activities related to collecting and reporting data is outlined below.

Activity	Timelinea
Data collection
Confirmation and completion of model profile data	August-September 2019
Collection of organizational chart and staffing information	September-October 2019
Telephone interviews	September – November 2019
Reporting
Model scan and telephone profile final report, includes updates from model profile confirmation	November 2019
Telephone interview summary document	December 2019

^a Subject to timing of obtaining OMB approval.

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

INSTRUMENT 1: State Model Profile

INSTRUMENT 2: Local Model Profile

INSTRUMENT 3: Request for Staffing Information

INSTRUMENT 4: State Telephone Interview Protocol

INSTRUMENT 5: Local Telephone Interview Protocol

References

Austin, James, and M. May Seitanidi. “Collaborative Value Creation: A Review of Partnering Between Nonprofits and Businesses: Part I. Value Creation Spectrum and Collaboration Stages.” Nonprofit and Voluntary Sector Quarterly, vol. 41, no. 5, 2012, pp. 726–758.

Bureau of Labor Statistics. “Usual Weekly Earnings of Wage and Salary Workers: Third Quarter 2018.” USDL-18-1661. Washington, DC: Bureau of Labor Statistics, October 2018.

Hulsey, Lara, Andrea Mraz Esposito, Kimberly Boller, Sarah Osborn, Rebekah Coley, Celina Kamler, and Jessica Zeigler. “Promise Neighborhoods Case Studies.” Princeton, NJ: Mathematica Policy Research, 2015.

Keast, R., K. Brown, and M. Mandell. “Getting the Right Mix: Unpacking Integration Meanings and Strategies.” International Public Management Journal, vol. 10, no. 1, 2007, pp. 9–33.

National Academies of Sciences, Engineering, and Medicine. Parenting Matters: Supporting Parents of Children Ages 0–8. Washington, DC: National Academies Press, 2016. Available at https://doi.org/10.17226/21868. Accessed June 1, 2018.

Ross, Christine. “Taskforce on Lifting Children and Families Out of Poverty: Coordinated Services for Parents and Their Children Memorandum.” Sacramento CA: AB 1520 Taskforce, 2018. Available at http://www.cdss.ca.gov/Portals/9/CalWORKs/
CA%20poverty%20task%20force_coordinated%20services%20memo_052218_for%20dissemination.pdf?ver=2018-05-23-153541-920. Accessed June 1, 2018.

Sama-Miller, Emily, and Scott Baumgartner. “Features of Programs Designed to Help Families Achieve Economic Security and Promote Child Well-Being.” Brief submitted to the U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research & Evaluation. Washington, DC: Mathematica Policy Research, September 2017.

1 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

11