Response to 60-Day Public Comments

Docket ID Number: ED-2021-SCC-0164. Agency Information Collection Activities; Comment Request; Evaluation of the Implementation of the Statewide Family Engagement Centers (SFECs)

This table below includes the comments received, related to the SFEC implementation study, in response to the public 60-day notice dated November 25, 2021, requesting comments on the Statewide Family Engagement Centers Implementation Study, 2018-2023 cohort, data collection activities.

Comment	Response
National Center for Family Learning (NCLF) tteater@familieslearning.org Public comment kyu-h8ou-4us8
	We thank NCFL for their comments and interest in the Department undertaking a study of the SFEC program. Below we provide responses to the specific recommendations NCFL made related to the SFEC implementation study.
1) Is this collection necessary to the proper functions of the Department? Yes, this is a necessary collection, the results of which have the potential to aid the Department in directing future family engagement priorities and resources effectively to meet the acute needs of families furthest from opportunity. This collection is squarely aligned with Education Secretary Dr. Miguel Cardona’s final discretionary grant priorities, including but not limited to Priority 2, Promoting Equity in Student Access to Educational Resources and Opportunities, which has a focus on evidence-based strategies that include “ongoing, robust family and community involvement.” It is NCFL’s experience that feedback from evaluations seeds innovation and ensures programming remains relevant as families and circumstances change. An evaluation of the SFEC grant program will illuminate strengths and identify areas in need of improvement that can guide the Department as it looks for ways to effectively establish, expand, and improve high-impact family engagement across the country.	We appreciate NCFL’s input on the necessity of the proposed collection and agree that the information from the study will help inform the Department’s decisions moving forward.
2) Will this information be processed and used in a timely manner? The results of this evaluation and implementation study will not inform priorities and decisions for the second grant competition, as it is currently open. It is NCFL’s hope that the study can be applied to future grant competitions and additional Department efforts to support the expansion of high-impact family engagement.	While the findings will not be available for the 2022 grant competition, the findings will inform the program and will be available to inform future competitions.
3) Is the estimate of burden accurate? No. It is our opinion that because of the gravity of family engagement, this study should be conducted with greater depth, and more specifically that families that participate in the SFEC program should be part of the evaluation. NCFL is concerned that the scope of the SFEC programs and their impact on families and student academic achievement will not be sufficiently evaluated through a 90 minute survey and hour-long interview of each SFEC director and a 30 minute survey and hour-long interview of a state education agency representative. Furthermore, given that this program is directly targeted to families, it is striking that their voices are missing from this evaluation. We must uplift the voices of families in all the work we do within the SFEC system to spur the most equitable and innovative practices and focus on shared learning and accountability. Thus, NCFL recommends interviewing parents to hear from them directly and to lift up the value their voices bring to all facets of this program. A key component of successful family engagement is elevating parent voices. Secretary Cardona has identified the need to incorporate parent voices in various places within the Department’s Final Priorities as a strategy to address equity, improve engagement, and advance systemic change. That includes “Establishing, expanding, or improving the engagement of underserved community members (including underserved students and families) in informing and making decisions that influence policy and practice at the school, district, or State level by elevating their voices, through their participation and their perspectives and providing them with access to opportunities for leadership ( e.g., establishing partnerships between civic student government programs and parent and caregiver leadership initiatives).	The burden estimate provided in the Supporting Statement was for data collection activities that are proposed for the study. We have adjusted the burden estimate based on the pilot testing of the survey and interviews with the SFEC and SEA representatives. While this comment references the burden estimate, the specific suggestions also speak to the scope of the study. The Department agrees that understanding family engagement is important and is developing a portfolio of studies that will use a variety of methods and sources of data to understand the processes and impacts of family engagement more fully. The SFEC program itself is a small program with limited resources. Thus, at this time, the Department is not planning full scale data collection for this implementation study. In planning future, full scale data collection about family engagement strategies the Department considers families an important source of data. The Department recognizes the value of including parents’ voices in successful family engagement. The SFEC program specifically requires that each SFEC establish a special advisory committee whose membership must include parents. We have developed additional response options to better understand the nature of the special advisory committee.
4) How might the Department enhance the quality, utility, and clarity of the information being collected? NCFL strongly recommends enhancing the information collection so that it can provide more depth in understanding the successes and challenges associated with the SFEC program. Parent engagement is a means to multiple “ends”– the most important being student success, which is intertwined with family well-being, social justice and equity, and school quality and performance. To deepen understanding of the effect of SFECs on its end goals, in addition to surveying and interviewing participating families, NCFL recommends enhancing research questions related to outcomes for children and families. Sample questions include: How does family engagement during COVID-19 build parenting adults’ capacity to partner with their child’s school? How has family engagement supported through the SFEC program impacted student academic achievement? What strategies implemented in the SFEC program supported families in a way that has resulted in meaningful positive change for children and families? How has the COVID-19 pandemic sparked innovation in implementation of high-impact family engagement?	The Department recognizes that to fully understand the process and impacts of family engagement strategies a range of research questions and approaches to answering them may be needed. The Department is conducting the current study as a first step. This is an implementation study of the SFEC program, not an impact study. The Department has chosen to conduct an implementation study at this time to better understand SFEC activities and how they are carrying out the family engagement. Investigating the impacts of the family engagement strategies are not in the scope of the current study. Because this is an implementation study, the study is limited to describing SFEC strategies. The study design does not allow for conclusions to be drawn about the effect of SFEC strategies on outcomes such as capacity to partner with schools; future IES studies may look at the effect of strategies beyond the SFECs, however, through designs that permit assessment of impact on outcomes. The study instruments include questions related to the impact of COVID-19 as well as how lessons learned about high-impact family engagement from 2020-2022 might be carried over post-pandemic. The role of COVID-19 is explored in section C of the SFEC Director survey instrument ( “The Impact of the Covid-19 Pandemic on SFEC Program Implementation”) and on the followup interview protocol, questions 2, 14, 15, 16, 17, 18, and 19.
National Parent Teacher Association (National PTA) No email provided Public comment kyu-ozso-ju07
	We thank the National PTA for their comments and interest in the Department undertaking a study of the SFEC program. Below we provide responses to the specific recommendations National PTA made related to the SFEC implementation study.
(1) This collection is necessary to the proper functions of the Department of Education. National PTA believes that this collection is necessary to the proper functions of the Department of Education, especially around its goals of meeting students’ social, emotional, and academic needs, improving academic achievement, promoting equity, and deepening family engagement in education.	We appreciate National PTA’s input on the necessity of the data collection related to family engagement and the SFEC program.
It is National PTA’s belief that the effectiveness, sustainability, and scalability of the Statewide Family Engagement Center program will hinge on solid data and unified evaluation metrics. Understanding the performance and impact of the program will illuminate best practices that can be replicated, identify opportunities for growth and improvement, and underscore the effectiveness of the program and the benefits of expansion into additional states.	We appreciate National PTA’s interest in both the implementation and impacts of the SFEC program. This implementation study does not include an investigation of the impacts of the SFEC program. If the Department conducts an impact study of the program it will seriously consider the collection of data from families.
(2) The results of this collection will be processed and used in a timely manner to inform Congress, current and future grantees, the Department of Education, and the public. National PTA anticipates the results of this collection will be processed and used in a timely manner by the Department of Education to refine requirements for the FY 2023 and FY 2024 grant cycles and to inform the work of current and future grantees by highlighting best practices and innovative strategies for family engagement. Moreover, the results of this collection can be used to broaden awareness of effective family engagement strategies, enabling community partners and families to replicate best practices in their own communities on a local scale. National PTA also hopes that the results of this collection will be used to inform decisions around federal budget and appropriations, especially for Fiscal Year 2024. Robust funding from federal appropriators will be needed to not only sustain the great work already underway, but also to expand the program to ensure that all families across the country have access to this program in the coming years.	The Department will use the findings from the study to inform future decisions about the program.
(3, 5) The estimated burden of this collection is accurate, and the Department can minimize this burden by providing grantees with clear and consistent guidance around deadlines and requirements and by utilizing digital data collection tools that are accessible. National PTA believes the estimated burden as detailed in Supporting Statement Part A SFEC 11.16.21 Section A.1211 is accurate and is reasonable given the importance of this collection and the SFEC program to families across the country. Only minimal effort would be needed to ease the coordination of surveys and scheduling.	The Department appreciates National PTAs input on the reasonableness of the estimated burden. Based on pilot testing, the burden estimates will be increased slightly.
National PTA believes the Department can minimize this burden by utilizing digital data collection tools that are accessible, like Calendly and online survey tools, to collect information from grantees. As the Department moves forward with this data collection, it is also crucial that measures are clearly defined to avoid them being widely interpreted by grantees, and that deadlines and requirements are clearly communicated to grantees.	Digital data collection will be used to collect survey data from the state education agency. Information collected from the SFEC directors will be tailored to the specifics of each SFEC. Given the discrete number of SFECs and the resources required to customize an online data collection tool, the SFEC survey will be administered via paper.
4) The Department of Education can enhance the quality, utility, and clarity of the information to be collected in this study by encouraging consistency and disaggregation, focusing on meaningful outcomes and impacts, identifying appropriate grantee contacts, collecting quantitative and qualitative data especially from parents, and ensuring results are accessible to families. It is National PTA’s belief that the effectiveness, sustainability, and scalability of the SFEC program will hinge on solid data and unified evaluation metrics. A unified evaluation is important to demonstrate how program funds are being used and the effectiveness of services and supports provided. 1.The evaluation should be unified so that all SFECs are collecting consistent data that can be aggregated across projects. 2.The data should be collected and compiled annually. 3.The evaluation data should demonstrate the total number of parents, education professionals and others served by the SFEC, broken down by demographics. 4.The evaluation should measure outcomes and impact of the services and support provided. 5.The unified evaluation data requirements should reflect the variety of SFEC services that different programs may provide but not be so specific that they become overly cumbersome.	The Department agrees on the importance of standardized data collection across respondents. Thus, the data collection instruments have been structured to collect similar information across all SFECs. The implementation study is not an annual data collection. SFECs do submit information on an annual basis to the Department in Annual Performance Reports (APRs). Those APRs also follow a standard protocol used by all SFECs. The current study is an implementation study, not impact study. Investigating the impacts of the family engagement strategies is not in the scope of the current study. The Department has chosen to conduct an implementation study at this time to better understand SFEC activities and how they are carrying out the family engagement. Rigorous testing of impacts of family engagement services more broadly may be undertaken later by the Department as part of its larger portfolio of studies designed to better understand strategies for family engagement. We agree that the SFEC services should not be too specific and cumbersome for respondents. We identified services by reviewing current grantee APRs to extract services that were common across grantees. The information in the APRs has informed the development of the survey instruments and variety of services, activities, and topics that are presented in the survey questions (for example, in Questions A1-A8) As suggested, information will be collected in this study to be able to describe the variety of services provided by SFECs.
Another critical area for improving the quality of this data collection will be for the Department of Education to ensure that an appropriate contact within each State Educational Agency (SEA) is identified to respond to any required survey or interview. Unfortunately, there has recently been a good deal of turnover at SEAs and, in some cases, SFECs have worked with multiple people in different departments since the start of their grant period. It would negatively impact the quality of this collection if, for example, the Department’s SEA contact on SFECs was only recently hired. National PTA recommends that the Department develop criteria for the contact person at each State Educational Agency for this grant and/or collect data to understand the historical relationship of who is being surveyed or interviewed from the SEA. It might also be worthwhile to encourage or require that the SEA identify a backup contact who has a consistent relationship with the SFEC in their state.	The Department agrees that is important to identify a respondent at the state level who can provide informed responses to the survey and interview questions. Based on instrument pretesting, we have revised the SFEC survey to specify that the primary SEA contact be the person “where most of the family engagement work is concentrated” and that this contact person is someone “with whom the SFEC has worked with the longest and/or works with the most frequently and knows the most about the SFEC’s work.” We will then confirm with the contact listed by the SFEC that they are the most knowledgeable to discuss the work of the SFEC, and if not, work with them to determine who the best respondent at the SEA.
As the Department moves forward with this collection, it is crucial that all evaluation measures are clearly defined to avoid them being widely interpreted by grantees. In addition to the data the Department plans on gathering, we believe that collecting qualitative data from grantees would create a more comprehensive and quality visual of their goals, objectives, progress, and accomplishments.	The study will collect data from SFEC directors and state education agency representatives through both surveys and interviews. It is expected that the interviews will solicit qualitative data that provides insight into the implementation of the SFECs. See the interview protocols for SFEC Directors and SEA representatives for the qualitative data this study will collect to explore goals, objectives, progress, and accomplishments.
Soliciting feedback directly from a diverse sample of parents and families who have utilized the SFEC program will also be essential as the Department seeks to identify areas of strength and growth for the program. We strongly recommend that evaluators engage in meaningful and frequent consultation with parents and families who utilize the program to gain a deeper understanding of how the program is fulfilling its purpose.	The Department is developing a portfolio of studies that will use a variety of methods and sources of data to more fully understand the processes and impacts of family engagement. The implementation study of the SFEC program is an initial step toward fully understanding the work of the SFECs. In planning its portfolio of studies on family engagement, collecting data from families will be important.
Finally, we encourage the Department to take steps to ensure that the results of this collection are accessible to families and disseminated to key partners.	The Department will take steps to make the findings from the study accessible to a range of audiences, including families. The final report will be written for understanding by a general audience. The Department will make the report available publicly (the report will be available on www.ed.gov) and will encourage the SFECs to share the report and findings with their stakeholders. The Department’s Family Engagement Team may also disseminate it to national parent and teacher organizations, and otherwise make it accessible to families in its usual manner.
National Center for Parent Leadership, Advocacy, and Community Empowerment (National PLACE) dautin@parentsatthetable.org Public comment kyu-o9hc-y5ld
	We thank National PLACE for their comments and appreciate their interest in understanding the implementation and impacts of the SFECs. We provide responses to their specific recommendations below.
Part A. However, in addition to surveying and interviewing the SFEC Director and SEA, National PLACE would also recommend surveying and interviewing a sampling of Special Advisory Council members, including parents, community-based organizations, and other partners. Their perspectives are critical to understanding the success – or lack thereof – of Statewide Family Engagement Centers. Part B: We would recommend adding the sampling of Special Advisory Council members, see above.	The current study is an implementation study, not impact study. The Department has chosen to conduct an implementation study at this time to better understand SFEC activities and how they are carrying out the family engagement work. Investigating the success of the family engagement strategies are not in the scope of the current study. The comments received from the public will inform the larger portfolio of evaluations on family engagement that the Department is planning. The scale of the study is limited at this time, and Special Advisory Council members – including parents, community-based organizations, and other partners – are not sources of data for the current study.
SFEC Director Survey Protocol Comments Technical assistance and infrastructure building activities, as defined in this survey, are activities that require the recipient to reach out to access the service, as opposed to being actively recruited to participate in the service. In general, these services are broadly disseminated so that a potentially unlimited number of recipients may obtain access. They may also require a time-limited amount of effort by the SFEC because the materials, once created, do not need to be repeated or tailored to multiple audiences. Some common examples of these activities include large conferences that any interested parties may attend, resources on websites, webinars, framework implementation guides, meetings with partner networks, and communities of practice. This definition is not consistent with our experience in technical assistance or infrastructure building activities including communities of practice. Communities of practice often include members who are actively recruited to participate in the service, and further, once created, the CoPs still need ongoing support and tailoring to multiple audiences. Further, true infrastructure building activities are not time- limited. We agree that large conferences that any interested parties may attend, resources on websites, and framework implementation guides are generally time-limited. As a member organization of family- led, family-serving organizations, that provides TA and infrastructure building activities to our members, I would not say that this definition is consistent with my understanding of TA or infrastructure building. Nor is it consistent with the US Department of Education’s own description of what “technical assistance” means; that definition includes three tiers – universal (which is what this paragraph is allegedly discussing), targeted, and intensive. National PLACE recommends (a) a different title for the description of “time-limited amount of effort,” and (b) reconsideration of what time-limited activities would be included here. Perhaps this could be redefined as Universal TA, and not include CoPs or even conferences.	In the pilot test of the instruments, respondents found the definitions and examples were sufficient to allow them to categorize their funds. We have moved the term “communities of practice” to be part of the direct services definition. Not every program at the Department is adopting these definitions. The comment references levels of TA that are a common model used in the centers funded under the Technical Assistance and Dissemination (TA & D) program, especially the Regional Comprehensive Centers Program, but they are not definitions that are incorporated into the SFEC program or the Notice Inviting Applications (NIA).
Direct services, as defined in this survey, are services that require both SFECs to actively recruit participation as well as recipients (families, schools, LEAs) to accept involvement. These services are not broadly disseminated but rather provided to a targeted set of recipients, with limited SEA assistance, to families, schools, and LEAs. The goal of direct services is to intensively support families, schools, and LEAs in enhancing their engagement within the context of specific, identified needs to improve student achievement and behaviors for school success. These activities may be time-intensive for the SFEC because they include, for example, face-to-face or virtual interactive parent communities; intensive trainings for school and LEA leaders, teams, and educators to implement specific family engagement interventions or strategies (such as supporting families in guiding homework); home visits to families; and training parents in family literacy. Direct services typically require more active engagement by the SFEC and customers than does technical assistance. National PLACE recommends that the title of this paragraph be revised, perhaps to Targeted and Intensive TA. Direct services are more often interpreted as meaning direct educational services, related services, etc. We would also recommend adding conferences and communities of practice to this paragraph focused on more targeted and intensive TA. These changes would also impact the headings for the survey questions themselves, as well as where some of the questions are listed. For example, providing the infrastructure to support greater collaboration on family engagement topics, should be moved to the Targeted & Intensive TA category. We support the questions listed under Direct Services but would rename that category, Targeted & Intensive TA.	The Department has an interest in understanding the extent to which SFECs are providing direct services. At this time, the Department does not plan to adopt for the SFEC program the tiered levels of technical assistance referenced in this comment.
For the question related to providing services to families to support parent well-being (e.g., providing adult education services, financial education training, parenting classes), we would recommend adding “peer/parent to parent support” which is a significant service to families that supports parent well-being everywhere that topic is an option.	This example has been added.
For the question related to conducting training for schools or districts to encourage family leadership to advocate for children or understanding the school or district system, we would recommend adding “and to encourage shared/participatory leadership,” such as that involved in the former IDEA Partnership’s Leading by Convening model/approach to meaningful stakeholder involvement.	The concept of shared/participatory leadership has been incorporated into the response option, which now reads “Conducting training for schools or districts to encourage family shared/participatory leadership (e.g., advocating for children or understanding the school or district system, encouraging shared/participatory leadership)”
For the question in RQ1.3, we would recommend taking out “educational choice” in the listing. In the list of topics under question A5, we would recommend moving l, Social-Emotional Learning/Social-Emotional Development, which is a high priority topic, up on the list. Also for topic f, Family-school communication and engagement around academic achievement, we would recommend either adding “and social-emotional development,” or adding this as a separate topic (family-school communication and engagement around social-emotional development.”).	Educational choice was the focus of a competitive preference priority in the NIA “… to provide families with the information and tools they need to make important decisions regarding the educational that is most appropriate for their children” that these SFECs responded to. Thus it is an important topic to investigate, and the study will retain this response option. An option for activities related to “social emotional learning/social emotional development” is included. The Department is interested in understanding specifically communications related to academic achievement, so will not add the social-emotional development to that option.
If you keep i, focused on engaging families of students with disabilities as a special group, we would recommend adding engaging families with LEP, families of color, and immigrant families as special groups in the same way.	The surveys are designed to collect information systematically across the SFECs. The list of topics was developed based on the activities that were reported by SFECs, and this was a topic/set of activities that APRs included. The types of activities reported in the APRs did not warrant expanding the list the additional categories suggested. However, we have revised the wording of what was row I (now row n) to read: “Providing supports for families of disadvantaged students, such as help understanding individualized learning or development plans (IEPs or IDPs)”
Most of the questions listed under A8 are not really about educational choice, rather, they are about school accountability. While we would recommend eliminating the “educational choice” language completely, if that is not possible, we would recommend changing it to “educational choice and school accountability.”	Based on other feedback, the survey question about the specific educational choice activities was removed.
In A9 we would recommend adding Strengthening Families as well as the CDC’s Whole School, Whole Child, Whole Family (formerly Coordinated School Health model).	This is now question A6. No change was made. The purpose of the survey is to ask all SFECs about activities that are reported in APRs, so that the study has systematic data for reporting. The purpose of the surveys is not to provide additional ideas of what the SFECs could be doing. The information in the APRs did not warrant the recommended additions. Survey respondents may add in other strategies or models in the open-ended question at this item and those responses will be coded.
For A12, we would recommend adding, “Collaboration with other family education and engagement organizations such as Parent Centers.” (We note that the competition requires grantees to partner with the Parent Center(s) in their state, and yet many of the current grantees have NOT even reached out to the Parent Center(s) in their state. There should be a specific question asked about this fundamental component of the application requirements.)	We have added an option for collaboration with other family engagement organizations and have included parent centers as examples. This is now question A9, row j.
We strongly support including the questions in A14 as these are often barriers.	These questions have been retained; this is now question A11.
In B1 and C3, we would add families of children in the juvenile justice system, families with limited literacy (even though they are English speakers), and families where the parents themselves have disabilities.	The options were not revised, as they align with the categories of students that are named in the NIA. Further, the specific characteristics of parents are not mentioned in the NIA.
We recommend explicitly adding Parent Centers to the list in C4b.	We have included a new option for “Collaboration with other family education and engagement organizations and included parent training and information centers and community parent resource centers.” This is now question A9, row j.
SFEC Director Interview Protocol Comments We would recommend changing the title, Technical Assistance and Infrastructure Building, to Universal TA, and the title, Direct Services, to Targeted and Intensive TA.	As noted in response to other comments, at this time the Department is not adopting these tiers of technical assistance for the SFEC program. We have revised the “Technical Assistance and Infrastructure Building” title to no longer include “Infrastructure Building.”
Another potential reason to in question 2 about the reasons for the different focus now (the only prompt is the pandemic) should be input from the Special Advisory Committee. In fact, the role and activities and impact of the Special Advisory Committee should be reflected as a separate category in both the survey (see A12) and the interview, including in Question 3 as well.	We have revised row a of question A9 row a to read Special advisory committee meeting discussions. The interview protocol (SFEC Q4) will probe on the role of special advisory committees.
SEA Survey & Interview Guide We strongly recommend adding “social-emotional development” to the definition of family engagement (“involving student academic learning, social-emotional development, and other school activities.”) We would also take out “as appropriate” in this paragraph. Being included in decision-making and on advisory committees is never “inappropriate.”	We have revised the definition of family engagement to include “social-emotional development” and we have also removed the term “as appropriate.”
We make the same recommendations in terms of terminology/definitions in this survey as we did in the SFEC Director survey. We also make the same recommendations in terms of the lists of topics for this survey as we did for the SFEC Director survey.	We have applied all revisions made to the SFEC Director survey to the SEA Survey.
Question 8: If you are going to ask the SEA about their level of satisfaction with their partnership with the SFEC, National PLACE recommends also asking this question of the SFEC Director about their SEA. We also recommend adding the opportunity for both the SEA and the SFEC Director to provide additional information on WHY – whether it is why they are satisfied, or why they are not.	While the SFEC is required to partner with the SEA, the SEA is served by the SFEC. As such, we are asking the SFECs about their satisfaction with the SEAs.

5