Supporting Statement A - AMCS Virtual Site Visits OPRE Pilot with OIRA_Sept 2020 Extension

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Assessing Models of Coordinated Services for Low-Income Children and Their Families (AMCS) Site Visits

OMB Information Collection Request

0970-0535

Supporting Statement

Part A

SEPTEMBER 2020

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Laura Hoard and Ivelisse Martinez-Beck

Part A

Executive Summary

• Type of Request: This Information Collection Request is for an extension of an approved information collection (0970-0535). We are requesting one year of approval.

• Description of Request: This information collection is to inform ACF’s understanding of the day-to-day operations of state and local-level coordinated services approaches. Specifically, we will be conducting descriptive case studies to collect information about innovate approaches to coordinate early care and education with services aimed at family economic security and/or other health and human services at the state and local levels. We will collect information on up to six (6) coordinated services approaches through semi-structured interviews with approach leadership and staff, and focus groups with parents. We do not intend for the data we collect in the study to be generalized to a broader population. We do not intend for this information to be used as the principal basis for public policy decisions.

A1. Necessity for Collection

There are no legal or administrative requirements that necessitate this collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose

Purpose and Use

The purpose of the Assessing Models of Coordinated Services for Low-Income Children and Their Families (AMCS) project is to understand how states and local communities are coordinating services across sectors to most efficiently and effectively serve low-income children and their families. Policymakers and program leaders across the country are experimenting with innovative approaches to combine early care and education, family economic security, and other health and human services (Hulsey et al. 2015; Ross 2018; Sama-Miller and Baumgartner 2017). These approaches vary along a range of dimensions, including their number and types of partners, funding streams, target populations, goals and objectives, locations, services provided, and monitoring processes. This qualitative study aims to fill gaps in our knowledge by identifying and describing the features of state and local approaches to coordinating early care and education services with family economic security and/or other health and human services.

The information collected under this request will improve federal, state, and local policymakers’, practitioners’, and other stakeholders’ knowledge and understanding of approaches to coordinating services to support low-income children and their families. Information from these activities will be used to inform ACF leadership and the field about the characteristics of state and local coordinated services approaches that currently operate in the United States and provide technical assistance for Preschool Development Grant Birth to Five (PDG B-5) grantees. The PDG B-5 legislation states that the purpose of the grant is to “assist States to develop, update, or implement a strategic plan that facilitates collaboration and coordination among existing programs of early childhood care and education in a mixed delivery system across the State designed to prepare low-income and disadvantaged children to enter kindergarten and to improve transitions from such system into the local educational agency or elementary school that enrolls such children” (Section 9212 of the “Every Student Succeeds Act (ESSA), PL 114-95”). As States develop, update and implement coordinated services approaches in line with the PDG B-5 grant, it is imperative that they have information about best practices and lessons learned from existing coordinated services approaches. This study will provide information on existing coordinated services approaches that can be used to inform the work of the PDG B-5 grantees.

Research Questions or Tests

The study has six primary research questions (Table 1). These questions cover the full range of data collection activities of the study; the highlighted column shows the activities included in this information collection request. Information collected as part of the formative generic request (201904-0970-003) will inform the data collection that is part of this full information collection request.

Table 1. Research questions and sources of information

* Note: the size of the check mark denotes how intensely the data collection methods will be able to answer the research question.

Research questions	GenIC (201904-0970-003)	Current ICR (0970-0535)
	Model profile data Staffing information Telephone interviews	Site visits
1. Are coordinated services approaches able to coordinate partnerships and service application and delivery? Can we identify key characteristics of these approaches?		
2. How do coordinated services approaches intend to reduce barriers and road blocks for families to access services? Are there federal barriers to implementing such approaches?		
3. Are approaches that combine ECE, family economic security, and/or other health and human services able to address other child development factors beyond ECE?		
4. What have we learned from efforts to integrate enrollment and eligibility processes for health and human services?		
5. Are states and/or localities examining service delivery dynamics across ECE programs to assess availability of care slots and services to meet the needs of eligible families? How are they using data to understand service delivery dynamics?		
6. How is public and private ECE funding targeted to meet the needs of at-risk children and families? Are there differences in the families that are able to access services?		

ECE= early care and education.

Study Design

The AMCS study has a descriptive, qualitative design.

The project team will “visit” six (6) coordinated services approaches operating at the state or local level. Data will be used to produce individual site visit descriptions, as well as a summary report with themes from across the site visits.

Data will be collected through “virtual site visits”, including semi-structured interviews with approach leadership and staff, and focus groups with parents served by the coordinated services approach. Interviews will focus on topics critical to understanding how coordinated services approaches operate, such as partnership building, funding, service coordination and delivery, barriers and challenges to coordination, and lessons learned.

The project team will purposively select coordinated services approaches for the site visits. The coordinated services approaches will represent a range of approaches operating at the state or local level. More details about site selection are available in Supporting Statement B.

Our descriptive data collection has limits. We will collect as much information as possible, but “virtual visits” will be limited to the equivalent of 1.5 days of interviews with each site (i.e. 6-8 interviews lasting up to two hours). We can only capture a descriptive picture of what coordinated services approaches are doing and what they have learned about the work; we cannot measure or evaluate the effectiveness of the coordinated services approaches. We will take care in the presentation of findings to ensure they are interpreted as descriptive in nature and that they do not speak to the effectiveness of approaches. The findings of this ICR cannot be generalized beyond the approaches that participate in the data collection for this project. Despite these limitations, collecting in-depth qualitative information about a set of approaches will provide important information to inform technical assistance efforts and future studies. As interest in coordinated services approaches grows, it is important to gather information about how approaches operate and to understand staff experiences in coordinating service delivery. This information can serve as important context and foundation for future data collection efforts, which might include implementation, evaluation, or impact studies.

Table 2 provides information about each of the activities for this study, including activities covered under the generic information collection and the current information collection (shaded in grey). These activities build on one another: the state and local model profiles covered under the generic information collection provide a framework for thinking about the ways in which states and localities coordinate services. That information will be used to select state and local approaches to participate in telephone interviews (GenIC) and “virtual site visits”. Information collected under the generic will serve as context for “virtual site visits” and allow “virtual site visit” interviews to probe for more in-depth information about partnership processes, model features, barriers to coordination, and other key aspects of the state and local coordinated services approaches.

Table 2. Study design components and timeline

Study activity	Time frame	Respondent (s)	Content and Purpose	Mode and Duration
Model scan	October 2018–June 2019	N/A	Content: Broad search of publicly available information on approaches that coordinate ECE, family economic security and/or other health and human services. Purpose: Describe the general landscape of coordinated services approaches currently operating across the country.	N/A
Model profile data	October-December 2019 (GenIC)	One staff member from individual state and local coordinated services approach	Content: Model profile information. Purpose: Confirm and complete state and local model profile data.	Mode: Email and follow-up calls Duration: 2 hours
Staffing information	June-September 2020 (GenIC)	One staff member from individual state and local coordinated services approach	Content: Information about staff respondents for the telephone interview and organizational structure. Purpose: Provide contextual information for telephone interviews	Mode: Email and follow-up call Duration: 30 minutes
Telephone interviews	June-September 2020 (GenIC)	Staff from 20 individual state and local coordinated services approaches	Content: Information that extends model profile data across topics, such as development of the approach, coordination with partners, data systems and use, and services provided. Purpose: Understand features of coordinated services approaches and inform technical assistance and selection of sites to visit.	Mode: Interviews Duration: 1 hour
Mater site visit interview protocol	Current ICR extension, Winter 2020 / early 2021	Staff from six individual state and local coordinated services approaches Staff include: Leaders: Executive directors, program directors, CEOs, or staff in similar roles. Directors of services within the site: staff who may lead particular components (e.g., a director of early childhood services) that are within the overall coordinated approach. Frontline staff: staff who work directly with clients to provide services (including service navigators or coordinators).	Content: Detailed information about day-to-day operation of approach, including discussions with staff. Purpose: Develop an in-depth understanding of implementation.	Mode: Virtual interviews via secure audio/video conferencing technology Duration: 45 mins – 2 hour individual interviews Note: We will not administer the master interview protocol in its entirety, instead the project team will select and ask questions that are relevant to each respondent and coordinated services approach.
Parent focus group protocol	Current ICR extension, Winter 2020 / early 2021,	Parents (10-12) from each individual state and local coordinated services approach	Content: Discussions with families receiving coordinated services. Purpose: Develop understanding of coordinated service receipt experience.	Mode: Virtual focus groups via secure audio/video conferencing technology Duration: 1 hour

The data collection procedures for study activities are included in Supporting Statement B.

Other Data Sources and Uses of Information

Information gathered through data collection activities covered under the generic information collection (201904-0970-003) will be used as context for the site visits under the current extension of the approved information collection request. To reduce burden on participants, activities under the generic information collection leveraged information found in the public realm.

A3. Use of Information Technology to Reduce Burden

The most efficient way to collect most of the required information is to interview coordinated services approach staff and parents. However, whenever possible, information technology will be used in data collection efforts to reduce burden on staff. For example, outreach and scheduling will be done by email and interviews will be audio recorded to capture details from the interviews.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

The study will not collect information that is available from alternative sources. At each stage of the study, the study team will review notes from current activities to ensure that they do not duplicate information that can be reliably obtained through other sources, such as directly from the state or local approach website or from prior study data collection activities. For example, prior to “virtual site visits”, the study team will review the model profile information and telephone interview responses and only ask questions in the “virtual site visit” interviews that have not previously been answered as part of the model scan or telephone interview.

A5. Impact on Small Businesses

Some of the agencies included in the study will be small organizations, including community-based organizations and other nonprofits. The study team will minimize burden for respondents in the “virtual site visits” by streamlining interview questions to restrict interview length to the minimum required. In addition, the study team will be sure to conduct interviews at times that are convenient for the respondents.

A6. Consequences of Less Frequent Collection

This is a one-time request for information. Collecting these data are necessary for ACF to gain a better understanding of how states and local communities coordinate services to serve families, how they coordinate funding and service delivery, and the barriers to and facilitators of these functions.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on February 5, 2019, Volume 84, Number 24, page 1740-1741, and provided a sixty-day period for public comment. A copy of this notice is attached as Appendix 1. During the notice and comment period, 3 comments and one request for instruments were received, which are attached as Appendix 2. The comments were considered and detail was added in this package to describe what information would be collected and how that information would be used. In accordance with PRA regulations for the request of an extension of a previously approved information collection, an additional Federal Register Notice announcing the agency’s request for an extension was published on July 10, 2020, Volume 85, Number 133, page 41586-41587, and provided a sixty-day period for public comment. A copy of this notice is attached in Appendix 3. During the notice and comment period, 1 comment was received. The comment did not warrant any changes to the package.

Consultation with Experts Outside of the Study

Throughout the study, we have consulted with stakeholders and experts in the field. The stakeholders and experts represent the interests of the study at various levels, including federal, state, and local policymakers and program operators. Experts provided input on project products, including interview protocols, and will offer information about existing models that might be appropriate for data collection. To date, the study team has consulted with experts within ACF, including leadership and staff from the Office of Child Care, who have helped to shape and advise on the study activities. We have also consulted with five coordinated services experts: Betina Jean-Louis (Harlem Children’s Zone); Anne Mosle (The Aspen Institute); Teresa Eckrich Sommer (Northwestern University); Peter Tatian (Urban Institute) and Kristin Bernhard (Ounce of Prevention).

A9. Tokens of Appreciation

“Virtual site visit” interview data are not intended to be representative in a statistical sense. However, at each site, it is important to speak with participants with a range of background characteristics to capture a variety of possible experiences with the site’s coordinated services approach. Without offsetting the direct costs incurred by respondents for attending interviews, such as arranging child care or phone/internet access, the research team increases the risk that only those individuals able to overcome the financial barriers will participate in the study, affecting the quality of the resulting data and insights.

The interviews with coordinated services approach leadership and staff, and focus groups with parents, will all take place virtually via secure audio/visual conferencing technology. To offset costs of participation in the focus groups, we plan to offer parents a $25 gift card. The token of appreciation will be provided at the time of the focus group, after careful explanation of the focus group procedures. Any individual who chooses not to participate after receiving the detailed explanation will be provided the token so that the token is not perceived as a coercion to participate.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

We will collect interview and focus group participants’ names for use during recruitment and data collection. This personally identifiable information will be destroyed after completion of the interviews.

Information will not be maintained in a paper or electronic system from which it is actually or directly retrieved by an individual’s personal identifier.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information.

Data Security and Monitoring

The Contractor developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ personally identifiable information. The Contractor shall ensure that all of its employees, who perform work under this contract, are trained on data privacy issues and comply with the requirements outlined in the Data Safety and Monitoring Plan. All of the Contractor’s staff sign the Contractor’s confidentiality agreement when they are hired.

As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system; establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor must submit a Data Safety and Monitoring plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access.

A11. Sensitive Information ¹

There are no sensitive questions in this data collection. Respondents can refuse to answer any question they do not wish to answer.

A12. Burden

Explanation of Burden Estimates

Estimates for the information collection burden for each of the instruments are in burden table below. Burden estimates include time for respondents to review instructions, search data sources, complete and review the responses, and transmit or disclose information. We expect the total annual burden to be 420 hours. See sections A2 and B2 for details about each instrument and respondents.

Estimated Annualized Cost to Respondents

We expect the total annual cost for respondents to be $14,868.00 for the information collection in the current request. Average hourly wage estimates for deriving total annual costs are based on Current Population Survey data for the second quarter of 2020 (Bureau of Labor Statistics 2020). For each instrument included in the burden table, we calculated the total annual cost by multiplying the annual burden hours and the average hourly wage. For respondents, we used the median usual weekly earnings for full-time wage and salary workers age 25 and older with a bachelor’s degree ($35.40 per hour). We divided weekly earnings by 40 hours to calculate hourly wages.

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Instrument 1: Master Site Visit Interview Protocol	180	1	2	360	$35.40	$12,744.00
Instrument 2: Parent Focus Group Protocol	60	1	1	60	$35.40	$2,124.00
Estimated Annual Burden Total				420		$14,868.00

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

The total cost for the data collection activities under this current request will be $149,059. This amount includes all costs related to study design, development, field work, analysis and dissemination.

Cost Category	Estimated Costs
Instrument Development and OMB Clearance	$43,706
Field Work	$73,471
Publications/Dissemination	$31,882
Total costs over the request period	$149,059
Annual costs	$149,059

A15. Reasons for changes in burden

The burden that OMB initially approved in October of 2019 (0970-0535) has not yet been used. As such, no additional burden is included in this request. Rather, ACF is requesting to continue information collection using the previously approved burden.

A16. Timeline

The study team will invite state and local coordinated services approaches to participate in a “virtual site visit” over a two to three month period following OMB approval. Site selection will take place following the confirmation and completion of model profiles and telephone interviews covered under the generic information collection. After each “virtual site visit”, the project team will prepare a short summary of the set of interviews. The summary will describe the “virtual site visit” activities and briefly describe the key services in the coordinated services approach. All “virtual site visit” summaries are expected to be completed within one month of each visit.

Information collected may be incorporated into documents or presentations that are made public. In sharing findings, we will describe the study methods and limitations with regard to generalizability and as a basis for policy.

The tentative timeline for activities related to collecting and reporting data is outlined below.

Activity	Timelinea
Data collection
Semi-structured interviews with leadership and staff (virtual)	October – December 2020
Focus groups with parents	October – December 2020
Reporting
Site visit summaries	December 2020 – January 2021
Site visit summary document	March 2021

^a Subject to timing of obtaining OMB approval.

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

INSTRUMENT 1: Master Site Visit Interview Protocol COVID-19 Additions

INSTRUMENT 2: Parent Focus Group Protocol COVID-19 Additions

APPENDIX 1: 60 Day Federal Register Notice

APPENDIX 2: AMCS Public Comments

APPENDIX 3: Recruitment Flyer COVID-19 Additions

APPENDIX 4: 60 Day Federal Register Notice (Extension)

APPENDIX 5: AMCS Public Comment (Extension)

1 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

9