Supporting Statement A - AMCS Site Visits OPRE Pilot with OIRA_May 2019_Final v3

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Assessing Models of Coordinated Services for Low-Income Children and Their Families (AMCS) Site Visits

OMB Information Collection Request

0970-NEW

Supporting Statement

Part A

SEPTEMBER 2019

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Laura Hoard and Ivelisse Martinez-Beck

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a new information collection. We are requesting one year of approval.

• Description of Request: This information collection is intended to inform ACF’s understanding of the day-to-day operations of state and local-level approaches to coordinate early care and education with services aimed at family economic security and/or other health and human services. Specifically, we propose to conduct descriptive case studies to collect information about innovative coordinated services approaches. We will collect information on up to six (6) coordinated services approaches through semi-structured interviews with coordinated services approach leadership and staff, and focus groups with parents. We do not intend for the data we collect in the study to be generalized to a broader population. We do not intend for this information to be used as the principal basis for public policy decisions.

A1. Necessity for Collection

There are no legal or administrative requirements that necessitate this collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose

Purpose and Use

The purpose of the Assessing Models of Coordinated Services for Low-Income Children and Their Families (AMCS) project is to understand how states and local communities are coordinating services across sectors to most efficiently and effectively serve low-income children and their families. Policymakers and program leaders across the country are experimenting with innovative approaches to combine early care and education, family economic security, and other health and human services (Hulsey et al. 2015; Ross 2018; Sama-Miller and Baumgartner 2017). These approaches vary along a range of dimensions, including their number and types of partners, funding streams, target populations, goals and objectives, locations, services provided, and monitoring processes. This qualitative study aims to fill gaps in our knowledge by identifying and describing the features of state and local approaches to coordinating early care and education services with family economic security and/or other health and human services.

The information collected under this request will improve federal, state, and local policymakers’, practitioners’, and other stakeholders’ knowledge and understanding of approaches to coordinating services to support low-income children and their families. We will produce individual site visit descriptions, as well as a summary report with themes from across the site visits, which will serve as a source of well-documented information about the characteristics of state and local coordinated services approaches and the strategies they are implementing to help support low-income families. Information from these activities will be used to inform ACF leadership in order to set program and evaluation priorities. Findings will also be used to provide technical assistance for Preschool Development Grant Birth to Five (PDG B-5) grantees, who are currently developing, updating and implementing coordinated services approaches.

The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.

Research Questions or Tests

The study has six primary research questions (Table 1). These questions cover the full range of data collection activities of the study, including activities currently in process as approved under the overarching generic: Formative Data Collections for ACF Research (0970-0356; 201904-0970-003). The highlighted column shows the activities included in this information collection request (ICR). Information collected as part of the generic information collection (GenIC) inform the activities under this full information collection request.

Table 1. Research questions and sources of information

* Note: the size of the check mark denotes how intensely the data collection methods will be able to answer the research question.

Research questions	GenIC (201904-0970-003)	Current ICR
	Model profile data Staffing information Telephone interviews	Site visits
1. How do state and local coordinated services approaches coordinate partnerships and service application and delivery? What are the experiences of those involved with these coordinated services approaches?		
2. How do coordinated services approaches intend to reduce barriers and road blocks for families to access services? Are there federal barriers to implementing such approaches?		
3. Are approaches that combine ECE, family economic security, and/or other health and human services able to address other child development factors beyond ECE?		
4. What have we learned from efforts to integrate enrollment and eligibility processes for health and human services?		
5. Are states and/or localities examining service delivery dynamics across ECE programs to assess availability of care slots and services to meet the needs of eligible families? How are they using data to understand service delivery dynamics?		
6. How is public and private ECE funding targeted to meet the needs of at-risk children and families? Are there differences in the families that are able to access services?		

ECE= early care and education.

Study Design

The AMCS study has a descriptive, qualitative design.

The project team will visit six (6) coordinated services approaches operating at the state or local level.

We plan to collect data through in-person site visits, including semi-structured interviews with approach leadership and staff, and focus groups with parents served by the coordinated services approach. Interviews will focus on topics critical to understanding how coordinated services approaches operate, such as partnership building, funding, service coordination and delivery, barriers and challenges to coordination, and lessons learned. Focus groups will focus on the experiences of families who receive coordinated services.

The project team will purposively select coordinated services approaches for the site visits. The coordinated services approaches will represent a range of approaches operating at the state or local level. More details about site selection are available in Supporting Statement B.

Our proposed descriptive data collection has limits. We will collect as much information as possible while on site, but visits will be limited to 1.5 days each. We can only capture a descriptive picture of what coordinated services approaches are doing and what they have learned about the work; we cannot measure or evaluate the effectiveness of the coordinated services approaches. We will take care in the presentation of findings to ensure they are interpreted as descriptive in nature and that they do not speak to the effectiveness of approaches. The findings of this ICR cannot be generalized beyond the approaches that participate in the data collection for this project. Despite these limitations, collecting in-depth qualitative information about a set of approaches will provide important information to inform technical assistance efforts and future studies. As interest in coordinated services approaches grows, it is important to gather information about how approaches operate and to understand staff experiences in coordinating service delivery. This information can serve as important context and foundation for future data collection efforts, which might include implementation, evaluation, or impact studies.

Table 2 provides information about each of the proposed activities for this study, including activities covered under the generic information collection (GenIC) and the current information collection (shaded in grey). These activities build on one another: the state and local model profiles covered under the generic information collection provide a framework for thinking about the ways in which states and localities coordinate services. That information will be used to select state and local approaches to participate in telephone interviews (GenIC) and site visits (current request). Information collected under the generic will serve as context for site visits and allow site visit interviews to probe for more in-depth information about partnership processes, model features, barriers to coordination, and other key aspects of the state and local coordinated services approaches.

Table 2. Study design components and timeline

Study activity	Time frame	Respondent (s)	Content and Purpose	Mode and Duration
Model scan	October 2018–June 2019	N/A	Content: Broad search of publicly available information on approaches that coordinate ECE, family economic security and/or other health and human services. Purpose: Describe the general landscape of coordinated services approaches currently operating across the country.	N/A
Model profile data	August-September 2019 (GenIC)	One staff member from individual state and local coordinated services approach	Content: Model profile information. Purpose: Confirm and complete state and local model profile data.	Mode: Email and follow-up calls Duration: 2 hours
Staffing information	September-October 2019 (GenIC)	One staff member from individual state and local coordinated services approach	Content: Information about staff respondents for the telephone interview and organizational structure. Purpose: Provide contextual information for telephone interviews	Mode: Email and follow-up call Duration: 30 minutes
Telephone interviews	September-November 2019 (GenIC)	Staff from 20 individual state and local coordinated services approaches	Content: Information that extends model profile data across topics, such as development of the approach, coordination with partners, data systems and use, and services provided. Purpose: Understand features of coordinated services approaches and inform technical assistance and selection of sites to visit.	Mode: Interviews Duration: 1 hour
Mater site visit interview protocol	Current ICR, expected Winter 2020	Staff from six individual state and local coordinated services approaches Staff include: Leaders: Executive directors, program directors, CEOs, or staff in similar roles. Directors of services within the site: staff who may lead particular components (e.g., a director of early childhood services) that are within the overall coordinated approach. Frontline staff: staff who work directly with clients to provide services (including service navigators or coordinators).	Content: Detailed information about day-to-day operation of approach, including discussions with staff. Purpose: Develop an in-depth understanding of implementation.	Mode: In-person interviews Duration: 45 mins – 2 hour individual or small group interviews (up to 3 staff) Note: We will not administer the master site visit interview protocol in its entirety, instead the project team will select and ask questions that are relevant to each respondent and coordinated services approach.
Parent focus group protocol	Current ICR, expected winter 2020	Parents (10-12) from each individual state and local coordinated services approach	Content: Discussions with families receiving coordinated services. Purpose: Develop understanding of coordinated service receipt experience.	Mode: In-person focus groups Duration: 1 hour

The data collection procedures for study activities are included in Supporting Statement B.

Other Data Sources and Uses of Information

We will use information gathered through data collection activities covered under the overarching generic: Formative Data Collections for ACF Research (0970-0356) as context for the site visits under the current request. To reduce burden on participants, activities under the generic information collection leveraged information found in the public realm.

A3. Use of Information Technology to Reduce Burden

The most efficient way to collect most of the required information is to interview coordinated services approaches staff and parents. However, whenever possible, we will use information technology to reduce burden on respondents. For example, we will do outreach and scheduling by email and will audio record interviews to capture details from the interviews.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

The study will not collect information that is available from alternative sources. At each stage of the study, the study team will review notes from current activities to ensure that they do not duplicate information that can be reliably obtained through other sources, such as directly from the state or local approach website or from prior study data collection activities. For example, prior to site visits, the study team will review the model profile information and telephone interview responses and only ask questions in the site visit interviews that have not previously been answered as part of the model scan or telephone interview.

A5. Impact on Small Businesses

Some of the agencies included in the study will be small organizations, including community-based organizations and other nonprofits. The study team will minimize burden for respondents in the site visits by streamlining interview questions to restrict interview length to the minimum required. In addition, the study team will be sure to conduct interviews at times that are convenient for the respondents.

A6. Consequences of Less Frequent Collection

This is a one-time request for information. Collecting these data are necessary for ACF to gain a better understanding of how states and local communities coordinate services to serve families, how they coordinate funding and service delivery, and the barriers to and facilitators of these functions.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on February 5, 2019, Volume 84, Number 24, page 1740-1741, and provided a sixty-day period for public comment. A copy of this notice is attached as Appendix 1. During the notice and comment period, three comments were received. Based on one of the comments, detail was added in this package to describe what information would be collected and how that information would be used. The comments and ACF’s action in response are provided in Appendix 2.

Consultation with Experts Outside of the Study

Throughout the study, we have consulted with stakeholders and experts in the field. The stakeholders and experts represent the interests of the study at various levels, including federal, state, and local policymakers and program operators. Experts provided input on project products, including interview protocols, and will offer information about existing coordinated services approaches that might be appropriate for data collection, such as potential sites for site visits. To date, the study team has consulted with experts within ACF, including leadership and staff from the Office of Child Care, who have helped to shape and advise on the study activities. We have also consulted with five coordinated services experts: Betina Jean-Louis (Harlem Children’s Zone); Anne Mosle (The Aspen Institute); Teresa Eckrich Sommer (Northwestern University); Peter Tatian (Urban Institute) and Kristin Bernhard (Ounce of Prevention).

A9. Tokens of Appreciation

Site visit interview data are not intended to be representative in a statistical sense. However, at each site, it is important to speak with participants with a range of background characteristics to capture a variety of possible experiences with the site’s coordinated services approach. Without offsetting the direct costs incurred by respondents for attending interviews, such as arranging child care or transportation, the research team increases the risk that only those individuals able to overcome the financial barriers will participate in the study, affecting the quality of the resulting data and insights.

The interviews with coordinated services approach leadership and staff, and focus groups with parents, will all take place in person and during scheduled visits. To offset costs of participation in the focus groups, we plan to offer parents a $25 gift card. The token of appreciation will be provided at the time of the focus group, after careful explanation of the focus group procedures. Any individual who chooses not to participate after receiving the detailed explanation will be provided the token so that the token is not perceived as a coercion to participate.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

We will collect interview and focus group participants’ names for use during recruitment and data collection. This personally identifiable information will be destroyed after completion of the interviews.

Information will not be maintained in a paper or electronic system from which it is actually or directly retrieved by an individual’s personal identifier.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information.

Data Security and Monitoring

The Contractor developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ personally identifiable information. The Contractor shall ensure that all of its employees, who perform work under this contract, are trained on data privacy issues and comply with the requirements outlined in the Data Safety and Monitoring Plan. All of the Contractor’s staff sign the Contractor’s confidentiality agreement when they are hired.

As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system; establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor must submit a Data Safety and Monitoring plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access.

A11. Sensitive Information ¹

There are no sensitive questions in this data collection. Respondents can refuse to answer any question they do not wish to answer.

A12. Burden

Explanation of Burden Estimates

Estimates for the information collection burden for each of the instruments are in the burden table below. Burden estimates include time for respondents to review instructions, search data sources, complete and review the responses, and transmit or disclose information. We expect the total annual burden to be 420 hours. See sections A2 and B2 for details about each instrument and respondents.

Estimated Annualized Cost to Respondents

We expect the total annual cost for respondents to be $12,936.00 for the information collection in the current request. Average hourly wage estimates for deriving total annual costs are based on Current Population Survey data for the third quarter of 2018 (Bureau of Labor Statistics 2018). For respondents, we used the median usual weekly earnings for full-time wage and salary workers age 25 and older with a bachelor’s degree ($30.80 per hour). We divided weekly earnings by 40 hours to calculate hourly wages. For each instrument included in the burden table, we calculated the total annual cost by multiplying the annual burden hours and the average hourly wage.

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Instrument 1: Master Site Visit Interview Protocol	180	1	2	360	$30.80	$11,088.00
Instrument 2: Parent Focus Group Protocol	60	1	1	60	$30.80	$1,848.00
Estimated Annual Burden Total				420		$12,936.00

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

The total cost for the data collection activities under this current one-year request will be $149,059. This amount includes all costs related to study design, development, field work, analysis and dissemination.

Cost Category	Estimated Costs
Instrument Development and OMB Clearance	$43,706
Field Work	$73,471
Publications/Dissemination	$31,882
Total costs over the request period (one year)	$149,059

A15. Reasons for changes in burden

This is a new information collection request.

A16. Timeline

The study team will invite state and local coordinated services approaches to participate in a site visit over a two-month period following OMB approval. Site selection will take place following the confirmation and completion of model profiles and telephone interviews covered under the generic information collection. After each site visit, the project team will prepare a short summary of the visit. The summary will describe the site visit activities and briefly describe the key services in the coordinated services approach. We expect to complete all site visit summaries within one month of each visit.

Information collected may be incorporated into documents or presentations that are made public. In sharing findings, we will describe the study methods and limitations with regard to generalizability and as a basis for policy.

The tentative timeline for activities related to collecting and reporting data is outlined below.

Activity	Timelinea
Data collection
Semi-structured interviews with leadership and staff (in person)	March – April 2020
Focus groups with parents	March – April 2020
Reporting
Site visit summaries	April – May 2020
Site visit summary document	June 2020

^a Subject to timing of obtaining OMB approval.

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

INSTRUMENT 1: Master Site Visit Interview Protocol

INSTRUMENT 2: Parent Focus Group Protocol

APPENDIX 1: 60 Day Federal Register Notice

APPENDIX 2: AMCS Public Comments

APPENDIX 3: Recruitment Flyer

1 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

9