60D Comment Response

Responses to comments received for 1810-NEW, Docket number ED-2020-SCC-0092 during the 60 day public comment period.

SFEC TA: Main Comments Received on Online APR Form (August 2020)

Comment 1 from NCFL:

Directions for Explanations should direct respondents to specify the content of “other

opportunities.”

OESE Response:

This information collection does not include changing the GPRA measures. However, grantees are given the opportunity to provide supplemental information to document and explain specific “other opportunities” that are offered through their activities and services. Comment 2 from NCFL:

NCFL has submitted two midyear reports and one end of year report and has yet to receive feedback from the Department on the manner of its use. Further, timing of the

midyear and end of year reports do not correlate with school calendars. If the midyear report

was moved from May to June, better data would be available. End of year data includes actual

beginning of year school data (August and September) but quality and other variables are not

available until midyear the following year. The Executive Summary directs grantees to explain how previous evaluation data was used, thus it is imperative that the Department provide relevant feedback to grantees.

OESE Response:

OESE response: The Department has used this Annual Performance Report (APR) data to determine continuation awards. The timing of the reports will not be changed due to the proposed new reporting format and APR questions. The timing of the reports is tied to the performance periods of the grants. The executive summary asked grantees to explain how previous data was used by the grantee, not by the Department. The main purpose of grantee evaluations is to improve grantee performance.

Comment 3 from NCFL: The estimate of burden is an underestimate. The midyear and end of year reports result in double the time for data collection, analysis, and reporting. When all sources of data collection and reporting from the evaluator, project director, project partners, and finance department are tallied, the burden is roughly 200 hours for each of the two reports.

OESE response: Based on this comment on the comment and burden estimation submitted by the Ohio SFEC we are adjusting this burden estimate to 30 hours.

Comment 4 from NCFL: The Department could require ongoing reporting in departmental guidance by instructing grantees to track the number of participating parents in year one and onward to clarify participation over time.

OESE response: Requiring grantees to track the number of participating parents in year one and onward to clarify participation over time would add burden to the reporting which we are are unwilling to require. However, grantees are certainly welcome to track this information and share it with the Department.

Comment 5 from NCFL: The Department should avoid requiring grantees to use the G5 electronic site. Because the site only permits one passcode per grantee, all evaluation work must be manually transferred to the platform. Thus, the platform creates redundancy and promotes reporting errors. The Department should consider minimizing the midyear report, limiting it only to reporting on GPRA charts. Project Objectives can be thoroughly reported and explained in the end of year report.

OESE response: This proposed reporting system will allow the grantee to bypass the G5 electronic site for reporting APR data. Grantees already have flexibility in what data they report on the interim report. If the interim report provides adequate data to justify continuation awards many questions can be omitted by the grantee. Grantees tend to have the most trouble completing the GPRA charts for the interim report. So, the Department will not require those charts be completed for continuation awards.

Comment 6 from NCFL: The Department could make this collection more functional in terms of replication of effective programs if “high impact” and “leadership and capacity building” were defined.

OESE response: The Department will provide guidance for these terms in the APR guide.

Comment 7 from NCFL: The Department should consider engage current grantees in a collaborative forum to create definitions of terms based on their project designs and project objectives.

OESE response: The Department engaged nine current grantees to elicit feedback in the APR reporting forms and definitions in March of 2020. The Department is also planning to gather feedback from the current grantees on the APR guide after it has been used for the 2020 APR submission and make updates as needed.

Comment 8 from NCFL: The Department should identify novel and successful strategies used by grantees during the COVID-19 pandemic to collect information from families, schools, partners, and the SEA.

OESE response: Since we are attempting to clear a data collection form to be used for years to come we don’t want to add a specific question on this but this data can be reported by grantees in the open-ended section of the data collection instruments.

Comment 9 from NCFL: GPRA measure #3 could be improved by expanding to include a GPRA measure on student performance as a secondary impact of family engagement. This measure is necessary to show that SFEC activities are improving student achievement. Examples measures include a demonstrated decrease in chronic absenteeism, teacher reporting on improved engagement in the classroom, improved English proficiency, etc. This is a complex measure that addresses parent training, effective engagement, student academic achievement, support of learning in the classroom, activities at home or outside of school, and state and local decision-making. These variables should each be addressed through project objectives. Overall, the way this is organized, actual project quality is not aligned with adult, family, or student performance outcomes.

OESE response: This proposed data collection does not change the GPRA measures. We did discuss adding some additional GPRA measures at the March project directors meeting, but the Department has not made a decision on that yet.

Comment 9 from NCFL:

Parent engagement is a means to multiple “ends”– the most important being student success,

which is intertwined with family well-being, social justice and equity, and school

quality/performance. Key answers to this question should be reflective of how SEAs set forth

new policies to encourage, support, and enrich parent engagement, how LEAs are able to

garner SEA, federal, and community resources using the Department’s information, and how

SFECs are able to ensure sustainability after five years.

OESE response: SFECs are given the opportunity to provide supplemental information that is relective of of how SEAs set forth new policies to encourage, support, and enrich parent engagement, how LEAs are able to garner SEA, federal, and community resources using the Department’s information, and how SFECs are able to ensure sustainability after five years.

Comment 1 from Ohio:

The Executive Summary section of the proposed form does not function as an executive summary as most people understand them. The function of an executive summary to summarize a longer report to orient readers to an organization’s work for the given period of time. In the proposed form, this aspect is largely removed and replaced with short prompts requesting quantitative data relative to the GPRA measures, and some narrative responses relative to Competitive Preference Priorities. These responses do not serve to summarize the information presented in the larger report nor afford the respondent the opportunity to frame or contextualize their work. The Executive Summary section of the proposed form does not enhance the quality of information being presented nor reduce the burden on respondents.

OESE response: We originally chose this title because it is the title used in one the Departments APR forms. However, we agree it doesn’t accurately reflect the content in this section. We are changing this section to GPRA Data and Competitive Preferences.

Comment 2 from Ohio: The proposed form is a departure from how we have been completing annual reporting through the use of progress tables. The Project Objectives section of the proposed form seems to focus more on quantitative results, which makes it difficult to demonstrate success in areas where we defined that outcomes would be reported qualitatively. Further, the way that the Project Objectives section is organized by Project Objective followed by GPRA Measure, followed by performance measure may not align with what is presented in a respondent’s application narrative or logic model. This would create a major change to how a respondent organizes their reporting and may decrease the clarity and quality of information being collected and certainly does not reduce the burden on respondents when reporting. For example, many of our GPRA 2 goals are only really demonstrated to be successful using narrative text (e.g., created/maintained a council, established faculty partnerships to achieve a research-to-practice pipeline for family engagement practices in Ohio, worked with the SEA to create model district policies, etc.) and would no longer be captured fully in this section.

OESE response: The last question of the Project Objectives section asks grantees, “Have you provided complete data on your performance measures for the current grant year?” The response options are Yes and No. If they answer “No”, respondents are also asked, “If you have not provided complete data, when will the data be available and submitted to the Department? Please enter the date in the following format: MM/DD/YYYY.” We have changed this option to be formatted as an open response rather than a date. The new wording of the response option would be: “If you have not provided complete data or would like to provide a qualitative narrative regarding one or more of your performance measures, please enter that text here. Please also indicate by what date (MM/DD/YYYY) you will be ready to submit complete data to the Department.”.

Comment 3 from Ohio: The new section added to the proposed reporting form, Participating Schools and Districts, is highly problematic. As a statewide center, we offer services and support to all Ohio school districts, state agencies, and other state and regional organizations and prepare them to work with districts, schools, and families. We in many ways “work with” hundreds of Ohio districts. Without defining “working with” under this section, respondents may be left to report on 100s of districts. Further, it is inappropriate to ask for MOUs between Statewide Family Engagement Centers and schools or districts. This seems more appropriate as supplemental documentation because it is not representative of how many districts a center works with, nor was it required in the application or described in our narrative. The Participating Schools and Districts section of the proposed form is not necessary to the proper functions of the department, the estimate of burden presented by the Department is far lower than we as respondents estimate, does not enhance the quality or utility of the data collected, and does not minimize the burden to respondents.

OESE response: The Department will define “working with” in the APR guide that the Department is developing. This definition will narrow the number of schools and districts and reduce burden. The Department currently requires submission of MOUs; this data collection package does not change this requirement.

Comment 4 from Ohio: In the Partnership section of the proposed reporting form, it is unclear where a respondent would report in-kind funding and/or direct funding. This may reduce the accuracy of this information.

OESE response: The Department changed the reporting form to make it clear that the report would include in-kind and direct funding.

Comment 5 from Ohio: Generally, the proposed reporting form will not capture anything related to the charge of these Statewide Family Engagement Centers to improve and increase statewide infrastructure for family engagement nor the US Department of Education’s call for grantees to focus on the Dual Capacity Building Framework.

OESE response: The proposed reporting form will allow grantees to upload any additional relevant information related to the Dual Capacity Building Framework that a grantee deems relevant.

Comment 5 from Ohio: In sum, we prefer the current APR reporting form. The Ohio Statewide Family Engagement Center scored the highest of all applicants when applying for funding under the Statewide Family Engagement Center program and we would like the opportunity to share our success through the APR process. The proposed form does not cut down the burden of reporting time or improve accuracy of reporting. The areas for reporting are not aligned to the previous form and do not seem to be targeted at the correct level of analysis to describe the success of Statewide Family Engagement Centers relative to GPRA measures nor the spirits of or requirements described in the request for proposals.

OESE response: The current APR reporting form was always meant to be a temporary approach pending creation of a new data collection instrument. The purpose of the APR is to provide for uniformity of reporting and analysis across SFECs. The new APR form enables that goal. Furthermore, through optional document uploads, it offers opportunities for grantees to add qualitative information and narratives to clarify and explain their quantitative measures.

Comment 1 from National PTA:

National PTA supports the Department of Education’s proposal to standardize information collection for the Statewide Family Engagement Centers (SFEC) program’s Annual Performance Report (APR). We believe that this step is necessary to support consistent metrics to evaluate programmatic impact and outcomes. As the Department moves forward with this new form of data collection, it is crucial that the GPRA measures are clearly defined to avoid them being widely interpreted by grantees. In addition to the data the Department plans on gathering, we believe that collecting qualitative data from grantees would create a more comprehensive and quality visual of their goals, objectives, progress and accomplishments.

OESE response: The Department agrees with this comment. We are currently creating an APR guide that will clearly define the data to provide clarity to current grantees and stakeholders. The proposed data collection package also provides ample opportunity for grantees to report qualitative data.