Module 3 Comments

Module 3 Comments_FY16.docx

Community Services Block Grant (CSBG) Annual Report

Module 3 Comments

OMB: 0970-0492

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0970-0492 can be found here:

Document [docx]

Download: docx | pdf

Module 3- Entire

This report contains the Module 3 Content for the following comments: 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,29,30,31,32,33,34,35,36,37,38,40,41,42,43,44,45,46,47,48,49,50,51,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,96,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,123,124,125,126,127,129,131,132,133,134

Comment 1 ( Arizona Department Of Economic Security, Division Of Aging And Adult Services, Community Action Programs And Services (DAAS/CAPS) ; State in AZ)

Module 3: Module 3 – Community Level: (pages 28-50)

The State CSBG Office Concurs with the comments made by the Community Action Partnership. Of particular concern are the indicators including percentages or rates. These unclear as to what they are seeking to determine, and our Network also believes aggregating this data will have little validity.

Module 3, Section A: Supplemental Data: (page # 32)

The State CSBG Office concurs with comments made by the Community Action Partnership on this item “Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.”

Module 3, Section B: Community Level NPIs: (page # 37)

The State CSBG Office concurs that there may be low report response on this section due to the indicators reflect social indicators outside of the purview of many CAAs.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Module 3, Section B: Community Level NPIs - Data Entry Form.

In the following sentence, “agencies must provide a narrative justifying the need for the initiative”, change the word “justifying” to “describing” or remove the sentence.

Comment 2 ( Clarity, Impact And Performance Project Steering Committee; RPIC in ID)

Module 3: Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

This section is more inclusive and will include longer term initiatives; can demonstrate involvement in the community.

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

Listings address strategies and not outcomes; strategies can change more often than annually; collaborative partnerships have more impact than strategies. Module 3, Section B: Community Level NPIs: (page # 37)

How is the baseline determined for Infrastructure and Asset Building metrics?

Comment 3 ( Hill Country Community Action Association, Inc. ; Eligible Entity/Local Agency in TX)

Module 3: No Comment

Comment 4 ( Sacramento Employment And Training Agency (Seta) ; Other/Unknown in CA)

Module 3: No Comment

Comment 5 (Action For Better Community ; Eligible Entity/Local Agency in NY)

Module 3: The agency has two concerns regarding this proposed change. First, the section requires that the agency establish baseline rates or percentages that apply to the community of interest. These benchmarks may be difficult or impossible to obtain. Furthermore, as the agency reports on its community impact strategies relative to these rates, it is unclear the degree to which changes in these rates should be expected to respond to the initiative.
For example, a successful community change initiative might be overwhelmed by an external event, like an economic downturn. Conversely, a community change initiative that has not been successful might appear to be so because external factors have changed the benchmark positively, not the initiative itself. Finally, aggregating information like this from local, to state to federal is not numerically sensible or feasible. One might be able to aggregate these data in order to report that X% of change initiatives across the network focusing on employment had reduced unemployment rates – but such information would itself be largely uninformative and would also be subject to the issues described above.

Second, the reporting tool appears to give a status to the Collective Impact model without that model having been established by the network as uniquely successful relative to other forms of collaboration. It is one form of community change, but by no means the only form. By identifying it, the data reporting tool favors it over others, as agencies will be influenced to use Collective Impact because of its presence on the report. The strength of the network resides in local decision-making, including the nature of the partnerships that will support positive change in the community.

Module 3, Section A

Reporting accurately community level benchmarks is likely to be resource-intensive if not impossible in some cases. It is recommended that these benchmarks either be removed or made optional so that agencies can report them if they are readily accessible, AND if they are believe to be significantly influenced by the change initiative(s) operated by the agency (these two conditions are unlikely to be met).

Module3, Section B

Community change strategy data are incredibly burdensome to track with accuracy, particularly as most change initiatives require communication with multiple community partners. As very few community partners are required to engage in this type of reporting, the mere need to request information of them will in some cases be burdensome. It is recommended that these indicators be replaced with narratives that allow agencies to provide the level of the data readily available.

Comment 6 (Action For Boston Community Development Inc.; Other/Unknown in MA)

Module 3: * Module 3: Community Outcome Indicators Based on Population Rates. The pursuit of community-level change is self-evidently important to Community Action. However, there are several logical fallacies embedded in the statement of these indicators. As a result, the data they generate is likely to be of little or no use to individual agencies or to states. There are fundamental problems with indicators which use changes in population-level ratios as the measure (e.g., “percent increase in children who are kindergarten-ready” or “increase in adult literacy rate.”) These measures, which are appropriate for large-scale public health interventions and efforts of similar scope, are unlikely to produce useful information in the context of a single agency’s actions over one year. The group is aware of the OCS guidance which allows for individual definition of “community”—however, this is unlikely to eliminate the problem. If “community” is defined as a service population (which would be necessary in most cases in order to report meaningful outcomes), the measure is essentially reporting on the success of a particular CAA program; this is information that fits more appropriately under Individual and Family Level NPIs.

Comment 7 (Allegany County Community Opportunities And Rural Development, Inc. (Accord); Eligible Entity/Local Agency in NY)

Section B: Community Level NPIs: (page # 37) Remove the Community Outcome Indicators that include percent or rates. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50) Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.

Comment 8 (Ashtabula County Community Action Agency; Eligible Entity/Local Agency in OH)

Module 3: No Comment

Comment 9 (Association Of Nationally Certified Roma Trainers ; Eligible Entity/Local Agency in N/A)

Module 3: The community indicators asking for baseline numbers and percent of change are unclear. For example, if an agency helps to create a community center in a community where there is no such facility, the baseline would be zero and the percent would be calculated by dividing i by 0 —which is not a real number. Or if an agency helps to bring 30 new jobs to a community where
there are 3000 jobs but the unemployment rate is 20% (thus indicating that there is a need for MANY more jobs) what will the calculation of a percent demonstrate? We propose using a narrative for identifying need and then, as the community project progresses, indicate what
measurement will be used to identify success (l community center, 30 jobs, etc.) We should remove the "and percent" in the indicators. We support the use of three types of community work: independent, partnership, collective impact. This is a way to clarify the kind of efforts of the network. We feel the use of "other indicators" will be very useful to secure more report data from the field. Could there also be "other indicators" in the family level module?

Comment 10 (Blue Valley Community Action Partnership; Other/Unknown in NB)

Module 3: accomplishments. Finally, in rural areas using community impact can be very difficult to justify; first because no one can do it alone and second because total record keeping is usually done on a much wider scale than one community or even one county in some areas.

Comment 11 (California Community Action Partnership Association; Other/Unknown in CA)

Module 3: No Comment

Comment 12 (Cap Service ; Other/Unknown in WI)

Module 3: Module 3, Section A, Page 29: I support the removal of the Community Outcome Indicators that include percent or rates, consistent with the Partnership. Their inclusion is inappropriate and doubt will provide any meaningful or value-added data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or Community Action Partnership ROMA Next Generation Talking Points Memo 4
targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use. Please consider removing "and percent from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. The pool size can artificially skew
percentages, CAP Services may find it difficult to collect this data if it does not correspond to readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available).

Module 3, Page 30-31: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary and challenging as I described earlier. This should be removed and included in training and technical assistance endeavors. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone. Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address
"Rates", specifically: Education and Cognitive Development: 5, 6, 7, 10; Housing: 6, 7, 8; Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, ); Public Safety (4a, 4c) A more appropriate system may allow CAP Services to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn't yet resulted in outcomes). Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network for ongoing training, technical assistance, and general capacity building supports.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50). These instructions are misleading and are inappropriate. It will negatively impact the capacity of CAP to perform its functions properly, particularly given our strategic development work plan and priorities established by our needs survey.

Module 4, Section A: Characteristics for NEW Individuals and Households: (page # 52) Remove this report in its entirety. I do not believe it is not necessary for the proper performance of CAP Services, Inc., and when rolled up to the state, the information has no practical utility. At a state level it could/will be misinterpreted and can be influenced by what happens in larger population centers. Additionally, the Characteristics report is of little use to determine if the agency is responding to needs identified in its community assessment. Demographic data alone does not identify the needs the new customers may have and is therefore of limited utility in comparison to data about individual, family, and community needs. And, since several core CAA programs (e.g. LIHEAP) have a large number of new customers with limited or one time contact with the agency, demographic data collected from the
low touch" services would confuse meaningful analysis of demographic trends. Many CAAs have to rely on data collected by their partners and grantees (in the case of public agencies), many of whom may have limited capacity to collect demographic data for new individuals and households. Lastly, there are many practical problems to identifying who constitutes a new customer or household which also constitutes an additional data collection burden on staff. While CAP Services sub-contracts little or none of its CSBG funding, we have experienced challenges in collecting required data through other projects, particularly if a non-profit may be small or emerging. Should a need exist to do so, I can attest that some entities would prefer to pass on a contractual agreement simply because of the cumbersome reporting that would follow such as agreement.

Comment 13 (Cattaraugus Community Action, Inc.; Eligible Entity/Local Agency in NY)

Module 3: No Comment

Comment 14 (Center For Community Future ; Other/Unknown in N/A)

Module 3: MY COMMENT. Social indicators are not an effective or practical way to measure the results of a CAA (agency) or the programs (strategies) the CAA has selected. Social indicators belong only in the Community Assessment where they can help people understand conditions and focus attention on conditions. Most of the twenty-some social indicators listed in the NPI’s are based on the interactions of large numbers of people, but most CAA’s will have no connection with most of these people. Most social indicators listed are derived from tens to hundreds to thousands of factors, including population dynamics and even conditions and policies in other countries, and the CAA is not even aware of most of them. Most social indicators are obtained as reifications of social and economic and political dynamics which are only partially understood or are based on weak theories of social science. Most social indicators have no clear cut strategies associated with them that describe how to change that indicator. The CAA has no information about and no control over most of the people, factors, and dynamics that are reflected in a social indicator, therefore it is not an effective way to measure agency-level results. A few social indicators from health (vaccinations, teen pregnancy) have identifiable links with powerful strategies that can move the indicator, but they are exceptions and not the rule. Unfortunately, the treatment model and the “injection theory” of change still exists in community action. Community action, human development and most social services are not the same kinds of interventions as treating patients for whom there are clear-cut diagnoses of conditions that can be cured with injections. Community action is not a science- based strategy; it is a belief system based on social values, moral principles, good intentions and aspirations. The community action network debated the potential use of social indicators as measures of agency and program effectiveness during the development of previous planning, management and measurement systems -- and rejected the concept each time.

1967-68 during development of the Grant Application Process.
1979 -81 during development of the Grantee Program Management System.
1993-95 during the development ROMA.
CAA’s have also been active participants in other Federally-inspired efforts to measure change on a community-wide basis using social indicators, such as:
1973-75 Service Integrated Targets of Opportunity (SITO, with HEW and United Way).
1993—Empowerment Zone and Enterprise Community Initiative (EZ/EC, Urban and Rural).
These efforts do not validate the concept of using social indicators to measure agency level activity. Allowing agencies to change the elements that are in a social indicator but to still call it with the same name as the “standard” social indicator is a recipe for confusion. For example, allowing agencies to change the denominator to reduce the size of the population against which changes are being measured (i.e. from a city-wide measure to half the students in a classroom) changes the measure from a social indicator into – something else. A new local measure like that may indeed be useful, but do not label it with the same name as the social indicator because most people outside the CAA will have a different idea about what the social indicator with that name means. A few CAA’s are involved in community-level collaborations in which the entire collaborative seeks to change that which is measured by the social indicator. Providing a space in the CSBG IS for CAA’s to report their involvement (with that or any other collaborative effort) would be O.K., but that is a different kind of reporting system from one in which CAA’s have to reject or pass over a substantial number of NPI’s that seek to measure their agency and program results. We understand that CAA’s are not making change happen through collective impact that might be reflected in a social indicator. If more use of the collective impact strategy is desired, then start with a capacity-building effort to encourage and assist more CAA’s to try it. Start with capacity building, not with reporting. There are at least three other methods for creating and measuring agency-generated, community-level change that are better than using social indicators.
(1) The Office of Community Services funded development of the (community level) Success Measures Project at Neighbor Works. http://www.successmeasures.org/data-system
(2) The Community Services Administration funded development of Nonservice Strategies for CAA’s to use to produce community level change. http://www.cencomfut.com/NonserviceApproaches.htm
(3) Just ask the CAA. “How is your agency measuring the change you are trying to create?”

Comment 15 (Center For Community Futures; Other/Unknown in CA)

Module 3: Module 3 – Community Level: (pages 28-50)

MY COMMENT. Social indicators are not an effective or practical way to measure the results of a CAA (agency) or the programs (strategies) the CAA has selected. Social indicators belong only in the Community Assessment where they can help people understand conditions and focus attention on conditions. Most of the twenty-some social indicators listed in the NPI’s are based on the interactions of large numbers of people, but most CAA’s will have no connection with most of these people. Most social indicators listed are derived from tens to hundreds to thousands of factors, including population dynamics and even conditions and policies in other countries, and the CAA is not even aware of most of them. Most social indicators are obtained as reifications of social and economic and political dynamics which are only partially understood or are based on weak theories of social science. Most social indicators have no clear cut strategies associated with them that describe how to change that indicator. The CAA has no information about and no control over most of the people, factors, and dynamics that are reflected in a social indicator, therefore it is not an effective way to measure agency-level results. A few social indicators from health (vaccinations, teen pregnancy) have identifiable links with powerful strategies that can move the indicator, but they are exceptions and not the rule. Unfortunately, the treatment model and the “injection theory” of change still exists in community action. Community action, human development and most social services are not the same kinds of interventions as treating patients for whom there are clear-cut diagnoses of conditions that can be cured with injections. Community action is not a science- based strategy; it is a belief system based on social values, moral principles, good intentions and aspirations. The community action network debated the potential use of social indicators as measures of agency and program effectiveness during the development of previous planning, management and measurement systems -- and rejected the concept each time.

1967-68 during development of the Grant Application Process.
1979 -81 during development of the Grantee Program Management System.
1993-95 during the development ROMA.

CAA’s have also been active participants in other Federally-inspired efforts to measure change on a community-wide basis using social indicators, such as:

1973-75 Service Integrated Targets of Opportunity (SITO, with HEW and United Way).
1993—Empowerment Zone and Enterprise Community Initiative (EZ/EC, Urban and Rural).

These efforts do not validate the concept of using social indicators to measure agency level activity.

Allowing agencies to change the elements that are in a social indicator but to still call it with the same name as the “standard” social indicator is a recipe for confusion. For example, allowing agencies to change the denominator to reduce the size of the population against which changes are being measured (i.e. from a city-wide measure to half the students in a classroom) changes the measure from a social indicator into – something else. A new local measure like that may indeed be useful, but do not label it with the same name as the social indicator because most people outside the CAA will have a different idea about what the social indicator with that name means.

A few CAA’s are involved in community-level collaborations in which the entire collaborative seeks to change that which is measured by the social indicator. Providing a space in the CSBG IS for CAA’s to report their involvement (with that or any other collaborative effort) would be O.K., but that is a different kind of reporting system from one in which CAA’s have to reject or pass over a substantial number of NPI’s that seek to measure their agency and program results. We understand that CAA’s are not making change happen through collective impact that might be reflected in a social indicator. If more use of the collective impact strategy is desired, then start with a capacity-building effort to encourage and assist more CAA’s to try it. Start with capacity building, not with reporting.
There are at least three other methods for creating and measuring agency-generated, community-level change that are better than using social indicators.

(1) The Office of Community Services funded development of the (community level) Success Measures Project at Neighbor Works. http://www.successmeasures.org/data-system
(2) The Community Services Administration funded development of Nonservice Strategies for CAA’s to use to produce community level change. http://www.cencomfut.com/NonserviceApproaches.htm
(3) Just ask the CAA. “How is your agency measuring the change you are trying to create?”

Module 4 - Individual and Family Level: (pages 51-75)

MY COMMENT. If OCS and NASCSP want to find out what change(s) a CAA is helping to produce in individuals, rather than trying to do it across the entire CSBG network through ROMA Next Gen, then OCS should fund or require evaluations at the CAA level. DISCUSSION. For $25,000 to $50,000, a CAA could hire a university professor who would help them structure and manage a three- to five year evaluation of a program or strategy that would provide insights about, and maybe even evidence of, changes the CAA helped create. OCS could fund these evaluations as separate projects. OCS required and funded evaluations as part of the Demonstration Partnership Program awards in the late 1980’s. About 80 evaluations were done. Those evaluations helped microbusiness and family development strategies expand and go to scale.

OCS could simply require that CAA’s do an evaluation using their block-grant pass through money. OCS does not now require evaluations of what CAA’s do with block grant money. In 1979-81, CSA mandated as part of the nationwide Grantee Program Management System that every CAA must do an evaluation on one of their programs every three years, using money from their ongoing CSA allocation. It was a good requirement then, and it would be a good one now.

Comment 16 (Central Missouri Community Action ; Other/Unknown in MO)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section B: Community Level NPIs: (page # 37)
• Employment: The impact that CAAs can have on employment in the community is limited by other factors that community action does not have control over. Therefore, we do not feel many of these could be reported on effectively.

• Education and Cognitive Development: Many of these sound like individual outcomes. The unit of measurement is individuals/families. A community outcome example: “The percentage increase of the number of post-secondary opportunities available in the specified community.” As
cited in the Introduction to ROMA curriculum on page 39 and 40, community outcomes focus on access, infrastructure, available resources, and opportunities.

• Infrastructure: The first two are appropriate and useful. With indicator 3, it does not seem feasible for community action to put predatory lenders out of business to decrease the number. However, community action could create positive alternatives to predatory lending in the community.

• Housing: “maintenance through WAP” seems very similar to “improving the value or quality through WAP” since the purpose of WAP is to improve quality and energy efficiency. Maintenance through WAP would also improve quality and value.

• Health & Social/Behavioral: #1 is good. #2, there seems to be an indirect relation between service provided and the infant mortality rate. We believe community action has many indirect impacts, but we should only be held accountable for and report on the direct impact. An alternative could be: the increase in opportunities afforded to low-income mothers to ensure healthy babies. #3, b – not sure this is meaningful. Will the data differentiate between the positive unplanned pregnancies and the negative?

• Civic Engagement: This is appropriate, meaningful, and useful.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

• Page 38: Housing #4 – percent of what? It makes sense to say 10 shelter beds were established in the specified community. 10% shelter beds cannot be established. The percentage would fit with #5 for the percent increase in shelter beds. Not all need percentages to be meaningful.

• Page 38: #7 – can an agency take the credit for a 10% decrease in the rate of foreclosure with all the factors that might be affecting this decrease? Maybe the agency offers related life skills classes or programs, but other agencies and real estate trends have an impact as well. These rate trends should be examined, but we are unsure this is meaningful information for CAAs to be reporting in the annual report. Moreover, where can this specific information be obtained? In our community commons web data tool we can access foreclosure rates, but even the national CAP community commons web data tool does not include this indicator. Another example is home ownership rates. We can access the number of homeowners, but it is not broken down by income level. Data should be accessible to agencies if it’s an option to be reported. If very few are able to report on it, it’s not meaningful to aggregate.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

• Page 41: Education #2 asks for the number of children that are kindergarten ready in a community. This information does not seem to be readily available. In a brief search, this was not found in the community commons national or state data tool, the kids count data book, or the Missouri Dept. of Elementary and Secondary Education data site. If this data is out there, agencies need to know where to find it. If they have to conduct their own research, additional resources and funding will be needed for this level of expertise. Reporting baselines is an option and not all indicators must be used, but if many agencies are not going to be able to find the information, it could be a local indicator rather than a NPI.

• Page 45: Housing #1 – if baseline includes all current safe and affordable housing units (say 100), then in utilizing CSBG and other agency resources it is determined that a realistic target is 2. Therefore, it would be an expected percentage change of 2% from the baseline. If actual results are that 1 home is built, then the actual percentage change is 1% and the performance target accuracy is 50%. Is this useful or able to show the positive impact of community action? If outcomes are being compared to the baseline in the entire community, it may present our impact as minimal and community action will appear ineffective.

Comment 17 (Central Nebraska Community Action Partnership; Other/Unknown in NE)

Module 3: No Comment

Comment 18 (City Of Austin Health And Human Services ; State in TX)

Module 3: Module 3 – Community Level: (pages 28-50)
Module 3, Section A: Community Level Initiatives

On the whole, this section seems unnecessarily complex. What are the differences between all the various kinds of initiatives? It will be very difficult to keep all this straight and not report overlapping data since some initiatives may fit more than one category.

Module 3, Section B: Community Level NPIs:

The need for a landing page is unclear. This same data seems to be collected in the following pages. There is a significant increase in the amount of data that is being asked of CAA’s to complete in this section. In addition, many of the descriptions are unclear in terms of how we will be expected to gather this information and what documentation we will need to maintain to support it. Changes on a community level to many of these measures will be difficult to tie specifically to the work of a CAA.
Many of the issues included relate more clearly to the work done with other funding sources and organizations (such as law enforcement or criminal justice). In general, reporting percentage increases/decreases in many of these areas will increase the amount of time and energy required for agencies expected to gather, track and maintain all this data, which decreases the time to actually provide the services that could have an impact.

Comment 19 (Colorado Community Action Association; State Association in CO)

Module 3: Community Level: (pages 28-50)

The use of social indicators in ROMA reporting is concerning, as programs and services provided by local CEE’s do not operate in isolation and social indicators are impacted by numerous actions outside their scope and control. Inclusion of items such as community literacy rates, foreclosure rates, crime rates, high school graduation rates, emergency response time, etc. is inappropriate in the NPI’s (but is appropriate in the Community Needs Assessment/Community Action Plan as required by the CSBG Act, which is where these indicators should be discussed and prioritized as indicators of change). Even if CEE’s were able to narrowly define their “community”, this type of reporting becomes inappropriate for rolling up to the state or national levels, and collecting data from existing sources would not be accessible or at minimum, extremely challenging, and would be virtually impossible to verify at the State level and National level.

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

N/A

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

· Should “Data Management” be listed as an issue/domain?

Module 3, Section A: Supplemental Data: (page # 32)

· Should “Data Management” be listed as an issue/domain?

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

N/A

Module 3, Section B: Community Level NPIs: (page # 37)

· Remove community-wide social indicators as previously recommended.
Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

· Remove community-wide social indicators as previously recommended.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

· Remove community-wide social indicators as previously recommended.

Comment 20 (Colorado Department Of Local Affairs ; Other/Unknown in CO)

Module 3: No Comments

Comment 21 (Combined Community Action ; Eligible Entity/Local Agency in TX)

Module 3: No Comment

Comment 22 (Community Action Agency ; Other/Unknown in NE)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

This is hard to conceptualize since cannot seeing the effect of entry on one page to another.

Module 3, Section A: Supplemental Data: (page # 32)

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

Module 3, Section B: Community Level NPIs: (page # 37)

Will there be a space to put more than one “other”?

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)
These charts will take so much time to collect data. Concerned about baseline data. Many times baseline is not available or is not accessible within the time frames of this report.

Comment 23 (Community Action Agency Of Beaver County ; Eligible Entity/Local Agency in PA)

Module 3: The addition of social indicators poses a grave concern for us as many of these indicators are not within our control as CAAs and it would be difficult to make a strong case for correlation on a large scale between CAA activities and indicators such as decrease in police response time and/or decrease in teen pregnancy as just two examples. While I understand that CAAs would not have to report on these indicators if they did not feel that they were a fit for the organization, simply having them on the report could put an expectation in the mind of those reading the report that these are somehow indicators that CAA is responsible for and/or have the capacity to strong influence. This could set up unrealistic expectations on the part of those responsible for approving CSBG funding at the legislative level every year. I was pleased to see that these indicators have been removed from the report for now and am hopeful that they are not added back in to the final version of the report.

Finally, in rural areas using community impact can be very difficult to justify; first because no one can do it alone and second because total record keeping is usually done on a much wider scale than one community or even one county in some areas.

Comment 24 (Community Action Agency Of Butte County Inc. ; Other/Unknown in CA)

Module 3: There are several points to the proposed Annual Report that render its practical utility at the local level to little of what it is hoped for especially when compared to the cost of the effort to generate the results in the report. It is unclear whether social indicators are still being recommended for use in goal setting for which NPI's will then be used to collect performance data against those goals for the Annual Report.

This was the case until recently. The use of social indicators is inappropriate for these purposes.

"The use of social indicators is inappropriate for these purposes.”

Thele is a miss-match between using social indicators as descriptors and using them for goals to attain t} much programmatic efforts. What exists as social indicator data has been traditionally collected through established data collection means e.g. Census Bureau, State agencies of local jurisdictions etc. This data is used to describe the conditions in local communities. That data which the local agency will realistically be able to generate may not replicate the collection methods already established. Even if it does it is duplicative for purposes of demonstrating impact in a much smaller application for which the use of traditional social indicators is inappropriate. Social indicators are appropriate for community-level condition assessments.

The utility of the data collected may be of benefit but it all depends on the extent of shared understanding among the community within which the agency operates. But this differs from agency to agency. While ROMA Next Gen allows for local definition Of a social indicator that fact then negates the ability to use that data for any purposes beyond the local area. So utility of the data is primarily limited to the agency and community where it is generated. Little use is possible to other agencies in other communities. And no use of it could be made on a state or national scale. Therefore the utility of some of the data being collected is insignificant as part of a national reporting framework. Agencies can collect such unique program data with local partners as they agree to for their purposes without the added burden of providing it for less relevance via mandatory systems of reporting.

The proposed Annual Report includes data collected as a result of the use of Community Measures. These Community Measures need considerable work in order for the data to be collected to be of any use beyond the local level. Even at the local level the use of Community Measures has limited value in that it highlights individual accomplishment unique to an agency and community given that differ from community to community. It would seem to be more cost-effective to allow local data collection to occur and another mechanism devised for sharing such data at the national or other levels rather than attempt to design such additional costly features into a national reporting system.

Community Measures not commonly understood

While the intent is laudable to identify and showcase the ways in which CAA's have been involved in communities with other partners on common activities it is clear that the lack of commonly understand and shared factors in place will be an inefficient stumbling block for implementation much less utility of resulting data for years to come.

The value of data at the national level (to warrant the investment of time, resources and more) relies upon the clear and shared definition and understanding of data. When everyone understands what something means then it lends itself to utility. The body of Community Measures on which some of the proposed Annual Report relies are not clearly understood by the national network. It will result in the creation of a disjointed data that do not add up to larger outcomes.

At the local level the information may have utility as it can assist in management of efforts to achieve pre-established goals. Beyond the local level the utility of the infon•nation is, at best, unclear since it is produced as a result of vastly different measures at the local level. Any attempt at summarizing such data at the national level for use in anything beyond a tick count is spurious. For many services the definition of them can and does vary widely from agency to agency. Without a definition of a service that takes into account the different approaches used by local agencies to implement the services then local agencies are left to operate local programs in as efficient a manner as possible.

Likewise data that is left to local definition for meaning is then lost for any use in the ROMA system. It is unclear whether the role of social indicators has been limited to community needs assessments or whether it is still being promoted as a reasonable source of goal material if the local effort is made to "tailor" a definition that limits the application of that data and goal. While this approach maybe more easy to the local agency it is useless in 160king to applicability to a national network.

The proposed Annual Report can have greater utility if it omits measures and methods of data collection that are, as yet, without any uniform or common definition or understanding. In this case Collective Impact is still an emerging "method" that community arc using to varying degrees of success. The use of collective impact acknowledges that many institutions "touch" the lives of individuals in the course of their efforts to change their lives. Yet, there remains no "science" as 10 how to collect and measure individual contributions to that effort. While a scientific approach isn't totally necessary it is further evidence of the need and benefit for further work in the area of definition and dialogue before such an approach is promoted as it is in the proposed Annual Report. Further confusion will be the result if these measures remain as part of the support system for the Report.

Comment 25 (Community Action Agency Of North Alabama ; Other/Unknown in AL)

Module 3: No Comment

Comment 29 (Community Action Agency Of Southern New Mexico; State Association in NM)

Module 3: Module 3 – Community Level: (pages 28-50)
Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Response: Define the difference between partnership initiatives and collective impact initiatives. The terms appear to be redundant.
Module 3, Section A: Strategies Popup Windows: (page # 33-36)

Response: Will the report allow the addition of other strategies?

Module 3, Section B: Community Level NPIs: (page # 37)

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Response: The number and percentage (of what??), i.e.

“The number (and percent) of jobs maintained for people in the specified community.”

A percentage implies a numerator and denominator. What numbers are we using?

Pages 37, 38 and 39 A percentage is required in many of the NPIs and is very confusing?

How are we to get a baseline for most of these percentages?

Is this useful information? Will it be comparable across the county? Will it mean anything to our stakeholders without a clear definition of baseline data??

Comment 30 (Community Action Association ; State Association in NM)

Module 3: Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Response: Define the difference between partnership initiatives and collective impact initiatives. The terms appear to be redundant.

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

Response: Will the report allow the addition of other strategies?

Module 3, Section B: Community Level NPIs: (page # 37)

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Response: The number and percentage (of what??), i.e.
“The number (and percent) of jobs maintained for people in the specified community.”

A percentage implies a numerator and denominator. What numbers are we using?

Pages 37, 38 and 39 A percentage is required in many of the NPIs and is very confusing?

How are we to get a baseline for most of these percentages?

Is this useful information? Will it be comparable across the county? Will it mean anything to our stakeholders without a clear definition of baseline data??

Comment 31 (Community Action Commission Of Erie, Huron, & Richland Counties ; Eligible Entity/Local Agency in N/A)

Module 3: No Comment

Comment 32 (Community Action Committee Of Victoria, Texas; Eligible Entity/Local Agency in TX)

Module 3: No Comment

Comment 33 (Community Action Council ; Eligible Entity/Local Agency in KY)

Module 3: Module 3 – Community Level: (pages 28-50)

The implementation of community-level data reporting will require the Council to create a new database in order for staff to begin to track this information. The cost will be overly burdensome as it will include the need for additional staff to track and monitor this data, a database administrator to monitor the data, and additional staff to follow up on the outcomes.

The Council agrees with the Partnership in that a more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section B: Community Level NPIs: (page # 37)

It will be very difficult to quantify the impact on individuals based on community-level indicators. For example, the number and variety of factors that impact the number of jobs created and maintained within a community are such that it will be difficult to create an accurate measurement tool to track and validate this information.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

The Council agrees with the Partnership that “and percent” should be removed from ALL community level indicators. Using “percent” is confusing, negatively impacts the quality, utility and clarity of the data, and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look (misleadingly) like a minimal result when compared with the overall county unemployment rate.

The Council agrees with the Partnership that rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

This data will be the most useful in determining what CAAs are actually implementing to create change within the community, and this is a feasible amount of data to track and manage. However, some of the indicators related to education and cognitive development are difficult to quantify and track. Few CAA staff are qualified to make assessments about individuals’ and families’ social and behavioral health. For this data to be useful at a state or national level, new measurement tools would need to be created and staff would need to be trained on how to properly assess the information.

Comment 34 (Community Action Council Of Lexington-Fayette, Bourbon, Harrison And Nicholas Counties ; Eligible Entity/Local Agency in KY)

Comment 35 (Community Action Council Of Portage County; Other/Unknown in OH)

Module 3: No Comment

Comment 36 (Community Action Partnership ; National Partner in DC)

Module 3: Module 3: Community Level

Remove the use of all rates and percentages in Sections A and B. The use of rates (e.g. violent crime rates, infant mortality rates) for local agency reporting to the state CSBG Office on community-level work is an inappropriate use of social indicator data. Community level change is affected by a wide variety of factors and the use of such indicators for reporting success or impact by a local agency’s efforts will not provide data that is useful. The use of percentages for community work is also misleading and inappropriate for use in reporting on local agency performance.

Remove “Collective Impact” options from the reporting tool. This is an area best left to training and technical assistance because (1) the data has little utility aggregated to the state and national level; and (2) “Collective Impact” is only one strategy for doing community-level work and should not be privileged over other approaches.

Comment 37 (Community Action Partnership Of Kern ; Other/Unknown in CA)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current “term of art” that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, education, etc.); (2) the types of strategies employed (e.g. EITC, etc.); (3) the target population (e.g. all at-risk high school students in the area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

· Remove Collective Impact. See above.

· See comments later regarding Outcome Indicators

Module 3, Section A: Supplemental Data: (page # 32)

· Remove Collective Impact

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Remove “and percent” from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide health initiative may successfully draw 100 participants, but may look like (misleadingly) a minimal result when compared with the overall county health rates.

Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

Education and Cognitive Development: 5, 6, 7, 10 o Housing: 6, 7, 8
Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, ) o Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate.

Comment 38 (Community Action Partnership Of Orange County; Eligible Entity/Local Agency in CA)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Often, structured collective impact initiatives are broad with many partners and with objectives that require multiple years to determine outcomes. The value of an annual report, especially at the onset of a multi-year/multi-dimensional project can be misleading. A different method of reporting for collective impact needs to be developed or removed.

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

Remove collective impact

Module 3, Section A: Supplemental Data: (page # 32)

Remove collective impact

Module 3, Section A: Strategies Popup Windows: (page # 33-36)
No comments

Module 3, Section B: Community Level NPIs: (page # 37)
No comments

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Remove “percent reporting”. Using percentage often misrepresents the significance of an issue e.g. Orange County, California has one of the lowest unemployment rates in California, but in fact has the third highest number of unemployed persons in the State.

Because of the lack of resources for large scale programs, Community Action Agencies often have programs effectively serving small numbers of individuals. The program may be 100 percent effective, but statistically insignificant compared to the total population or need for a specific service.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

No comments

Comment 40 (Community Action Partnership Of San Luis Obispo County, Inc.; Eligible Entity/Local Agency in CA)

Module 3: I am going to include some specific comments from my planning staff who coordinate the IS and CSBG reporting to CA CSD.

· Regarding the Community Level Initiative Status (pages 40-50 of the Annual Report) we do not understand how we can identify a Baseline number and then report out our numbers comparing them to this baseline. The initiatives are so broad and the agency is a small piece to a bigger puzzle that contributes to some of these initiatives. For the current NPIs we report specifically on what our agency accomplishes and this proposed initiative section would require obtaining numbers outside of our agency. This could prove to be very difficult.

· Another concern is the "target community' that we must identify for the Community Level Initiative Status categories. I can't wrap my head around how we would do this. I think they're trying to make it easier for us to be able to report on a specific area, such as a "neighborhood", but how is that defined? How do you drill down to that level and obtain those types of numbers. (As an example, Homeless Services works with the People's Kitchen and their volunteers provide meals at Prado. We have been collaborating with them for a long time and they recently decided they do not want to provide us with their volunteer #s or hours. If an agency who we have a long standing relationship with won't even provide us with this data how can we get some of the other initiative numbers for this new way of reporting. I could be wrong, but I'm worried about it).

· Starting on page 37, Community Level National Performance Indicators Landing Page, it is proposed at this point that for some of these categories that we provide both a number and a percent. The number makes sense to me, but coming up with a percent can be challenging and may not be possible. [i.e. "The number (and percent) of safe and affordable housing units maintained through WAP or other rehabilitation efforts in the specific community." We can provide the number of houses that Energy assisted through WAP, but how do we come up with the percent?]

Comment 41 (Community Action Planning Council Of Jefferson County, Inc.; Eligible Entity/Local Agency in NY)

Module 3: – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

• Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current “term of art” that refers to a specific
set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

• Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by eligible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

• A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

• Remove Collective Impact. See above.

• See comments later regarding Outcome Indicators Module 3, Section A: Supplemental Data: (page # 32)

• Remove Collective Impact

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

• No Comments

Module 3, Section B: Community Level NPIs: (page # 37)

• No Comments

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

• Remove “and percent” from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the
overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

• Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

o Education and Cognitive Development: 5, 6, 7, 10

o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, )

o Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

• This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate.

Comment 42 (Community Action Program Associations For Community Action Agencies; State Association in WI)

Module 3: No Comment

Comment 43 (Community Renewal Team ; Eligible Entity/Local Agency in CT)

Module 3: Module 3 – Community Level: (pages 28-50)

a. Community Level Initiatives Home Page: (page # 29) - Remove Collective Impact from the reporting tool. Collective Impact is simply one set of strategies for doing community-level work. Giving priority to one approach when there are other options for doing community level work is inappropriate and not necessary.

b. Remove all rates and percentages. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

c. A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

a. Remove Collective Impact. See above.

Module 3, Section A: Supplemental Data: (page # 32)
a. Remove Collective Impact.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

a. Remove “and percent” from ALL community level indicators. Percent can lead to confusing interpretation and negatively impact the quality, utility and clarity of the data. This has the potential to lead to data misuse and misinterpretation. This may also prove difficult for agencies to collect corresponding data relative to a unit or level of analysis to compute percentages and analysis. Not being able to obtain comparable statistics/percentages may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

b. Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

* Education and Cognitive Development: 5, 6, 7, 10
* Housing: 6, 7, 8
* Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c,)
* Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

a. This instruction is misleading, inappropriate and creates a reporting burden. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate.

Comment 44 (Community Services Agency - Reno; Eligible Entity/Local Agency in NV)

Module 3: I have these questions regarding EMPLOYMENT below:

· For Employment #2: “Jobs maintained” needs a time frame threshold or explanation! Does this imply 180 days, 90 days, or a shorter time frame?

· For Employment #4: “Jobs maintained”, again, needs a time frame (see #2 above)

· For Employment #5: Would a single 401K program or a single paid vacation benefit be considered “benefit package”?

· For Employment #6: Not sure why this is listed? It is too generic and because of this, it would not carry much value in reporting!

Comment 45 (Del Norte Senior Center, Inc.; Eligible Entity/Local Agency in CA)

Module 3: Module 3 – Community Level: (pages 28-50)

The level of reporting required for community-level work will do nothing but discourage agencies, especially small ones like mine, from even attempting to implement any community campaigns. Having to deal with seven sections of data entry and reporting, which take 22 pages to explain, just so that I can run an EITC outreach campaign is ridiculous. CSBG does not provide the level of funding required to make the kind of impact these extremely ambitious reporting requirements are designed to measure.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

ALL NPI’s ask for increases in positive indicators or decreases in negative indicators. There is no room for reporting work that maintains resources at a consistent level. For example, if we are already operating a 39-unit low-income senior apartment complex, how do we get credit for maintaining that safe and affordable housing? It doesn’t look like we can get any recognition unless we INCREASE the number of units available. Sometimes it requires all the resources we have, including CSBG, just to maintain programs that prevent community resources from being worse than they are. Unless increased funding is made available, that is all that can be accomplished.

This reporting is actually going to penalize me for not increasing measures that I am currently being given credit for maintaining at consistent levels. That is not acceptable.

Comment 46 (Delaware Opportunities Inc.; Eligible Entity/Local Agency in DE)

o Education and Cognitive Development: 5, 6, 7, 10

o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, )

o Public Safety (4a, 4c)

Comment 47 (Department Of Community Service And Development ; State in CA)

Module 3: The Community Level Initiatives are a new and inventive way to gauge the collective efforts of Community Action. Community Action is a well-known member of collective impact initiatives in communities around the country. The Community Action network would not be successful without numerous partnerships and collaborative efforts that support the wide array of services to low-income individuals and families. While this is true, it is not clear how quantifying the nationwide efforts supports or furthers the mission of Community Action. It is important to know how many partnership and collaborative efforts exist within our network and we currently have a method for collecting this information through the existing National Performance Indicator structure. The additional benefits to collecting more in-depth information on the status of the Community Level Initiatives are yet to be seen.

Consideration should be given to expanding the current National Performance Indicator reporting structure. A recommendation would be to add reporting fields to collect the number of initiatives by type, removing the reporting on the status of the initiatives and any other qualitative data points associated. The status and qualitative elements contained within the proposed structures are only informational in nature and likely will not produce improved delivery of services. Another suggestion is to remove the Supplemental Data reporting under this section. Further, strong consideration should be given to removing the following Community Level NPls as they are subjective and/or hard to quantify and track:

The number and percent of parents/caregivers who demonstrate increased sensitivity and responsiveness in their interactions with their children.

The number and percent of individuals who increase their net worth. Percent decrease in unplanned pregnancies in the community. The number of activities designed to improve police and community relations within the specified community.

The number and percent decrease in barriers/negative conditions to having a healthy community.

The number and percent of individuals who achieve and maintain capacity to meet basic needs for 90 days.

Percent decrease in recidivism rate in the specified community.
Percent decrease in emergency response time measured in minutes. Percent decrease in rate of substance use in the specified community (e.g. cigarettes, prescription drugs, narcotics, alcohol).

Comment 48 (Department Of Housing And Community Development ; State in MA)

Module 3: •Phase in the Module 3 – Community Level section of the report over a longer period of time to allow for States/CAAs to align data systems and not rush through training of staff. Module 3 is a brand new report form and although we agree it’s a valuable addition to the outcome measurements used by our network, it requires significant work with vendors to update existing data systems to allow CAAs to report on it. An additional year or more would allow us room to better design our system modifications and provide more in-depth training to our network.

Comment 49 (Department Of Human Services ; State in AR)

Module 3: No Comment

Comment 50 (Department Of Social Services Office Of Community Services ; State in CT)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

O “Local Agency Data Entry Form – Example…Partnership (optional)/Collective Impact: Common Agenda established (CAA will select)” – What information is meant to be entered in the second column, the actual agenda? Is a narrative meant to be entered into the second column or may a CAA upload a document? Same question for remaining rows.

O In addition, are CAAs required to complete all rows for Collective Impact?

O Will “backbone infrastructure” be defined? How does it differ from first row information “…what role does each partner play?”

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

o “Employment Strategies (check all that apply)” – If a CAA counts the number of actual “Job fairs” conducted (example of ‘3’ given in the sample), is that considered to be 3 strategies? If the conduction of “Job fairs” is a strategy, then holding multiple job fairs is simply the implementation of that one (1) strategy. It is not clear if the first column entries are initiatives or strategies since the second column requests number of strategies within each checked initiative. Another example: “Employer Education on Low-Income Issues” – if the CAA provides such information to 20 employers in the community within one workshop meeting or mailing, is the number of strategies 20 or 1?

o “Infrastructure and Asset Building Strategies (check all that apply)” – As above, it is not clear whether the entries in column one are ‘strategies’ (as the heading suggests) or ‘initiatives’ as detailed within the entries: Community Reinvestment Act (CRA) Initiatives; Police/Community Relations Initiatives; Eliminate Asset Limit Initiatives; Anti-Predatory Lending Initiatives; Asset Building and Saving Initiatives, etc. The second column asks for “Total Number of Strategies Implemented per Initiative”. For example, Police/Community Relations Initiative is listed as a strategy; column two requests number of strategies per initiative. A CAA may be initializing a Police/Community Relations Initiative and include 2 or 3 strategies within that initiative such as 1) partnering with local police department establishing programs that allow police to engage with residents outside of law enforcement arena; 2) offer cultural awareness training to officers assigned to specific neighborhoods; and 3) host events such as officers visiting early childhood classrooms, Touch-a-truck type event (squad car), and community CPR trainings conducted by officers. Are the three listings above considered strategies within the initiative of Police/Community Relations; or are they activities within the strategy of Police Community Relations? It just is not clear whether column one lists strategies or types of initiatives; and is column two looking for a number of the checked strategy or number of strategies not entered in column one.

o “Health and Social/Behavioral Development Strategies (check all that apply)” – as above, is a weekly Farmers Market considered 1 strategy or are CAAs expected to add the total number of Farmers Markets conducted?

O “Civic Engagement and Community Involvement Strategies (Goal 3) (check all that apply)” – Since CAAs are required to prepare a Community-wide Needs Assessment, is it expected that this strategy always be checked? Or is this a strategy separate from the required comprehensive needs assessment for CAAs’ catchment area?

O If a CAA is implementing multiple strategies for an initiative, where may such strategies be identified?

Module 3, Section B: Community Level NPIs: (page # 37)

o “Percent” is not made clear in this section – “1. Number (and percent) of jobs created to increase opportunities for people with low incomes available in the specified community”; Page 40 (Data Entry Form) provides some insight to “percent” meaning expected change from baseline; actual change from baseline; and performance target accuracy. CAAs must enter a baseline described as “existing starting point used for comparison”. Comment is continued in “Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)” passage.

O “Health and Social/Behavioral (check all NPIs for which CAA has outcome to report)” – This table is inconsistent with the others as all but one do not indicate a requirement for a number and percent.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

o “Civic Engagement and Community Involvement” – typographical errors at numbers 2 and 3 “and/or”.
O “1. Increased number (and percent) of people with low incomes who support Community Action’s delivery of service and/or implementation of strategies to address conditions in the specified community.” – How is “support” defined?

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

o See reference to Page 40 in Module 3, Section B: Community Level NPIs: (page # 37) passage – For that example noted above (1. The number of jobs created…), is a starting point the number of open positions within the community or the number of actual jobs (filled or not filled) in the specified community? If the CAA and its partners create jobs and some or all of those positions are offered and accepted by individuals outside of the community and/or non-low income individuals, is the “opportunity” still increased from merely the creation of the jobs?

O “Education and Cognitive Development # 1 - The number of accessible and affordable education assets or resources added to the specified community” – Does this range from early childhood (pre-school) education through post-secondary college opportunities? Since #s 2 – 11 specify within the range, how valuable is this data item?

O What is the expectation for column” III.)”? For example, under the heading Employment #5 “The number of jobs in the targeted community with a benefit package”; if II.) Baseline equals 100 meaning there are currently 100 jobs in the specified community with a benefit package (open and filled), would III.) Target be 20 meaning either 20 new positions will be created with a benefit package and/or a benefit package will be added to 20 existing positions or is the Target 120 meaning 20 will be added to the baseline? It is not clear what to enter as the other columns are auto-calculated.

O Concern that since a Target Community is self-defined by each CAA, be it a catchment area/county or 5-block square neighborhood, data requested here may be rolled up for one State or nationally and the roll-up may not be appropriate given the wide range of “community” definitions.

o Another concern is the time burden for CAAs as planning/development staff are often tasked with Community Needs Assessment, Community Action Plan, Strategic Planning administration (though a Board-driven document, staff usually prepare the plan and provide training/technical assistance to the Board as well as provide updates), project planning, grant applications, reporting and perhaps Organizational Standard compliance. Some elements of RNG may be better suited for Community Assessment development/preparation: Employment (creation of jobs); Infrastructure; and Health and Social/Behavioral as most CAAs may have limited capacity to effect change in these areas (more contributing members than backbone of Collective Impact). Other components that should remain at the local level for analysis rather than be reported up may be: Civic Engagement and Community Involvement.

O Overall suggestion that Community Level projects should remain at the local level for analysis rather than be reported up. With CAAs responsible for Individual and Family Level reporting on services provided directly by the agency, there is the same expectation of data collection for Collective Impact even if the CAA is not a “backbone” agency of the project but a contributing partner. CAAs can be relieved of some of the data burden letting the “backbone” organization take the lead, focusing on their component of the project and analyzing data specific to their catchment area.

O What is the definition of a living wage job? As this is a subjective measure and cannot be easily quantified, this is an inaccurate measurement of poverty. Additionally, it has not traditionally been the role of the CAA to create and/or maintain jobs in the community, so how will this be measured other than via subjective means.

O How will WAP services be counted as maintaining a “safe and affordable” housing unit? WAP is for energy efficiency improvements, not rehabilitation, as stated by statute. Additionally, what constitutes “safe and affordable”? Often clients cannot pay their rent/mortgage.

O How will homelessness be measured as this population is transient at best? For example, Connecticut has “ended” veteran homelessness, but there may, in fact, be veterans who do not wish to find more permanent dwelling which are not counted in that number.

O Ultimately, there is a question regarding adding resources or maintaining resources. Traditionally, this has not been a CAA responsibility as they administer their own programs and address holistic services for clients to aid them on the path toward self-sufficiency. If CAAs are to now add/maintain resources, they will require additional funding in hopes of creating/maintaining these resources. Otherwise, this form of measurement is disingenuous at best and could potentially destroy the CAA structure at worst.

O In terms of health services, traditionally CAAs do not hire nurses or other medical professionals so this, again, is disingenuous and misrepresents what CAAs do when they provide services to clients. Furthermore, how will obesity reduction be measured? We do not require clients to give their weight at time of intake nor would they consent to such a measurement.

O Where does LIHEAP, the major funnel through which clients interact with the agency, fit in under these new measures? Reduction in energy burden/energy payment is a major driver toward increased freedom with income etc.

Comment 51 (Dickinson Community Action Partnership ; Other/Unknown in ND)

Module 3: inclusion is inappropriate and will provide meaningless data when aggregated beyond
individual initiatives. Such data has validity for geographic areas much larger in than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.

Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)
• Remove Collective Impact. See above.
• See comments later regarding Outcome Indicators.

Module 3, Section A: Supplemental Data: (page # 32)

• Remove Collective Impact.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Remove "and percent" from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the "percent" has several problems. First, even though agencies are allowed to define what constitutes "community', it may be difficult for agencies to collect this data if it does not correspond to a unit or level Of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) Which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency's outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce
development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part Of a needs assessment. Remove all indicators that address "Rates", specifically:

Education and Cognitive Development: 5, 6, 7, 10
Housing: 6, 7, 8
Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c,)
Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

• This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, "agencies must provide a narrative justifying...." is inappropriate.

Comment 53 (Division Of Sate Service ; State in DE)

Module 3: No Comment

Comment 54 (East Missouri Action Agency, Inc.; Eligible Entity/Local Agency in MO)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

The Community Outcome Indicators that include “and percent”, are inappropriate and will cause our programs to appear ineffective. Here at our agency, different programs, such as CSBG, Head Start, Housing, etc. use the same NPIs to report the outcomes for their individual programs, and all have their own baselines for those programs. It is virtually impossible to include a percentage, considering there will be several different baselines used to reach a percentage, which in turn, causes skewed results.

We, as an agency, are allowed to define “community”, but I feel that we may be forced to make comparisons between our agency’s outcomes and the overall rate of a particular community level indicator. For example, if we were to build two units of safe and affordable housing, using NPI: Housing #1, how minimal would that percentage be compared to the baseline number of already existing units in our “community.” I’m afraid that this may make our programs and outcomes appear ineffective.

Comment 55 (Florida Department Of Economic Opportunity ; Other/Unknown in FL)

Module 3: * One of the biggest concerns is the department’s capacity to obtain the required data. Currently, we are required to report on 300+ data points. The proposal would require reporting of 1,000+ data points. Our current systems and staff are limited. Many of the proposed data points are not currently being collected by our case management system. Technology that will track all of the required information can be costly.
* Future CSBG funds are not likely to be sufficient enough to help sustain such a complicated system and additional staff. OCS has proposed an on-line automated system for use by states and possibly local agencies.

* Our agency will have to continue with our current case management systems and then manage the data input to the proposed online system. This will cost us staff hours to train and manage.

Comment 56 (Four Square Community Action ; Other/Unknown in NC)

Module 3: No Comment

Comment 57 (Fresno Economic Opportunities Commission ; Other/Unknown in CA)

Module 3: a. A new worksheet requires an Ongoing Initiatives Summary, description of New Initiatives, and report of Completed Initiatives. The information requested is thorough and comprehensive. Tracking and reporting on our efforts in progress will require additional staff time and attention on a much more regular basis than is now required.

b. The Strategies Popup – Data Entry Form (Module 3, Section A) adds significantly to the information now requested in the NPI. Even so, it is more than likely that many of our agencies are conducting initiatives unique to them, initiatives that are not captured on the proposed ROMA NG grid. Our point is – there is no way to assure that all activities are somehow represented in the proposed NPI Landing Page. Reporting by the network should be unique to each entity. To attempt to condense the activities of all agencies into a pre-set list is antithetical to the purpose and mandate of the Economic Opportunity Act – in all the forms of its re-authorization.

e. Is there significant value in distinguishing community initiatives from individual/household initiatives? Given the fine distinctions between a numbers of the data cells, we think consolidation makes more sense, notwithstanding our introductory comments, above, on local control, local assessments, and locally-driven initiatives and solutions.

Comment 58 (Greater Bergen Community Action; Eligible Entity/Local Agency in NJ)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

“In situations where the baseline is not available, agencies must provide a narrative justifying the need for the initiative, as well as detail on how change was measured and tracked…”

This is important for the agency’s board to know, and they should be the only ones who decide if the initiative is justified, not the state. If this information is not being aggregated, why is it in this report?

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Asking for the number and percent, the number and percent increase or decrease, or an increase or decrease in the rate of, for example, the rate of homelessness, the foreclosure rate, negative conditions, etc., I think is setting us up for failure. The CSBG network cannot be held responsible for decreasing the homeless rate. Of course, our goal is to reduce poverty, but we know that it takes a community. We have been working for 50 years. Aside from the difficulties in aggregating information from vastly different communities, a low national rate or percentage change might be ammunition for our critics.

Comment 59 (Greater Bergen Community Action ; Eligible Entity/Local Agency in NJ)

Section A: Community Level Initiative Status Page: (page # 30-31)

“In situations where the baseline is not available, agencies must provide a narrative justifying the need for the initiative, as well as detail on how change was measured and tracked…”

This is important for the agency’s board to know, and they should be the only ones who decide if the initiative is justified, not the state. If this information is not being aggregated, why is it in this report?
Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Asking for the number and percent, the number and percent increase or decrease, or an increase or decrease in the rate of, for example, the rate of homelessness, the foreclosure rate, negative conditions, etc., I think is setting us up for failure. The CSBG network cannot be held responsible for decreasing the homeless rate. Of course, our goal is to reduce poverty, but we know that it takes a community. We have been working for 50 years. Aside from the difficulties in aggregating information from vastly different communities, a low national rate or percentage change might be ammunition for our critics.

Comment 60 (Hinds County Human Resources Agency ; Other/Unknown in MS)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current “term of art” that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31) Remove Collective Impact. See above.

See comments later regarding Outcome Indicators

Module 3, Section A: Supplemental Data: (page # 32) Remove Collective Impact

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

No Comments

Module 3, Section B: Community Level NPIs: (page # 37)

No Comments

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Remove “and percent” from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to
the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.
Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

· Education and Cognitive Development: 5, 6, 7, 10

· Housing: 6, 7, 8

· Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c,) · Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate.

Comment 61 (Illinois Association Community Action Agency ; Other/Unknown in IL)

Module 3: Module 3 — Community Level:

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current "term of art" that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by eligible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn't yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

• Remove Collective Impact. See above.

• See comments later regarding Outcome Indicators

Module 3, Section A: Supplemental Data: (page # 32)

Remove Collective Impact

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

• A strong case can be made that these proposed data elements would have questionable statistical validity and would almost certainly have very poor statistical reliability. That is to say, it is not at all clear that these data elements would measure what they purport to measure. Furthermore, if 100 people reported on these data elements, there would likely be very little agreement among them with respect to what they thought they had reported. As a result, aggregating these local data at the federal, or even at a state level, has the highly probable effect of knowingly creating misinformation at best and creating disinformation at worst. It is challenging to rationalize how allocating scarce CSBG resources for this purpose achieves a meaningful public good.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Remove "and percent" from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the "percent" has several problems. First, even though agencies are allowed to define what constitutes "community", it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency's outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address "Rates", specifically:

o Education and Cognitive Development: 5, 6, 7, 10 o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, ) o Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

o This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, "agencies must provide a narrative justifying... is inappropriate.

Comment 62 (Inca Community Service; Eligible Entity/Local Agency in OK)

Module 3: Section A: Community Level Initiatives Home Page: (page # 29)

· Remove all collective impact options from the reporting tool. This information is easily misused and misleading.
Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

· Every local agency is different and the needs in the community differ. It could hamper the effectiveness of the agency meeting the needs of their community.

Comment 63 (Indiana Housing And Community Development Authority ; State in ID)

Module 3: No Comment

Comment 64 (Iowa Community Action Association; Eligible Entity/Local Agency in IA)

Module 3: Module 3 — Community Level: (pages 28-50)

The Iowa Network feels that removing collective impact, partnership and independent approach options from the reporting tool is an appropriate request. We see the value of the push for more community work because we know we cannot alleviate the causes and conditions of poverty alone. Given that belief, collecting percentages and rates towards success is not the way to measure local agency performance. Community Action Agencies are required to meet the needs at the local level and those needs vary so greatly that creating the kind of system that fits us in a box or holds us accountable for many social issues is not appropriate. We don't achieve community outcomes alone nor are we solely responsible when the outcomes are not achieved.

In regards to tracking community level work, it is not necessary to delineate between collective impact, partnership or independent types of community work, we do not see the need to be specific about the approach being used but rather allowing agencies the flexibility to report on unique outcomes being achieved through the projects they pursue.

Module 3, Section B: Community Level NPIs: (page # 37-50)
agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available), therefore with no additional resources or support for the agency to obtain this level of data it is like an unfunded mandate.

EXAMPLE Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used as part of the needs assessment. We agree with the importance of Community Action Agencies' community work, however, some agencies receive minimal CSBG funding and having an expectation for them to successfully address any of the social indicators listed is unrealistic. It is likely unrealistic for the agencies that receive a good portion of the CSBG funding unless they are to receive additional resources. If in a rural community there is a need for jobs and the Community Action Agency provides job training and job placement for the one major employer in the service area and the employer decides to close the business, should the Community Action Agency be held liable? Data would give the appearance of failure and the agency could be seen as unsuccessful even though the work they did could not keep the employer in the area. And although agencies are only supposed to report on the NPls that are relevant for them, it makes Community Action look ineffective if no community level work is reported. The real or perceived belief could be that Community Action does not do community level work and that is not true. It is just not being captured by this proposed package.

Comment 65 (Kceoc Community Action Partnership; Eligible Entity/Local Agency in KY)

Module 3: General comment for this section: Look at each and every NPI and tell us how we are supposed to track and document these and give us the tools to do this.

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

*Process, process, process! We are CONSTANTLY looking at initiatives that we intend to achieve outcomes, but that doesn’t always work out. Plus, some projects go on for YEARS before there is an outcome. That is when we currently report, when we have an outcome. We have our own internal way of managing projects and their progress. As CSBG Director, having to report on progress every single quarter would be extremely difficult, time consuming, and stressful among the many projects we may be tracking. We are perfectly capable with keeping up with our own projects. Perhaps there is a concern that some agencies spend CSBG dollars and funds are wasted? This is not the case with KCEOC. We conduct very intensive research before trying to start a new initiative and pre-plan extremely well.

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)
* See Section A comments

Module 3, Section A: Supplemental Data: (page # 32)

* See Section A comments

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

*I think part of this could be pulled together to help agencies have their own internal project tracking system to reach better outcomes. I see this as more of a helpful tool, not a constant reporting feature. Again, because its process!

*Page 34, Housing Strategies-Need to add Community Housing Development Organization (CHDO); these too create new affordable housing. Also, not just ending chronic homelessness! Rural communities deal more with families. Chronic homelessness under HUD’s definition does not address family homelessness. Also, need to add Youth homelessness/runaways, housing preservation, creation & EXPANSION of shelters, and homeless outreach

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

* Under education & employment, ensure there are NPIs that are consistent with the new WIOA regulations

* Under civic engagement, etc. (page 39), how is this supposed to be documented? Define “support” from low-income people and how would you track and document this?

*We are only tracking increases under civic engagement goal 2?

*Under housing, #5 (page 38), we only report about the number increase of shelter beds maintained? What about stating “the number of shelter beds maintained or increased?”

Comment 66 (Lakes And Prairies Community Action Partnership ; Other/Unknown in MN)

Module 3: Module 3 — Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives: (page # 29-32)

Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of
the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

A more appropriate system may allow a local agency to report (l) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn't yet resulted in outcomes).

Module 3, Section A: Strategies: (page # 33-36)

While the list of strategies included is cumbersome, it is at the same time not inclusive of the work that is done at the community level. Many of the strategies that our agency implements at the community level would be recorded under the "other" category. It would be helpful if the strategies could be written in a way that they were more about the outcomes and less about the work that is accomplished. For example, "Strategies that increase living wage employment".

Module 3, Section B: Community Level NPIs: (page # 37-50)

Similar to the strategies listed in Section A, the list of Community Level NPI's is cumbersome, and at the same time not inclusive of the work that our agency does a the community level. For example, our organization has conducted extensive work in the area of housing for households experiencing a crisis at the community level, and all of our current efforts would be recorded under the "other" category. NPI's should be more focused on broad outcomes for more streamlined reporting. For example, "Strategies that increase access to safe, affordable housing," or "Reduction in the number of days households experience homelessness.”

In addition, it would be unduly burdensome for local agencies to track and report the amount and level of data proposed for each of the NPI's. It is unrealistic for local agencies to be required to report at this level and expect community level changes to occur in such a short time frame. In addition it is unrealistic to expect that all areas will continue to increase from the baseline, for example the % increase of donated time and resources to support Community Action's delivery of services.

Comment 67 (Lorain County Community Action Agency ; Other/Unknown in OH)

Module 3: No Comment

Comment 68 (Louisville Metro Community Service ; State in KY)

Module 3: Module 3 – Community Level: (pages 28-50)

· The entirety of Module 3 should be returned to OCS to rework with Community Action Agencies. Module 3 overcomplicates the work of CAAs in their local communities. OCS may consider allowing CAAs to report on on-going initiatives using the current National Goals 2 and 3, and simply designate how many initiatives are new and how many are continuing.

Comment 69 (Lycoming-Clinton Counties Commission For Community Action (Step), Inc.; Eligible Entity/Local Agency in PA)

Section A: Community Level Initiatives Home Page: (page # 29)

- No Comments - Makes sense.

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

- Status Update is a great idea to illustrate the amount of comprehensive community projects that are simultaneously worked on by a CAA.

- Collective Impact is not necessary language to use – it confuses the partnership model and is too formal to work on a local level. Collective Impact makes sense in theory, but in reality it will not be viable!

Module 3, Section A: Supplemental Data: (page # 32)

- Is this necessary when it is already in the Home Page?

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

-Under the Infrastructure and Asset Building Strategies – should include: Blight Elimination; Brownfield Revitalization Programs; Flood Mitigation Initiatives

-Under Housing Strategies – should include: Re-Entry Housing Initiatives

-Under Health and Social/Behavioral Development Strategies – should be Developing, Maintaining, or Supporting Community Health Clinic

Module 3, Section B: Community Level NPIs: (page # 37)

- For the Community Level National Performance Indicators – it does not make sense to have number and percent for many of the indicators. The percent becomes difficult to calculate as it is the percent of all jobs in a community – how would this be calculated? How would you calculate the percent increase – is it from what it was for the whole community? For these, the number should just be used.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

-Under Education and Cognitive Development – 3. And 4. Will be extremely difficult to report on as a CAA.

-For percent increase in graduation rate – how can CAA state that it is their outcome, as many factors contribute to this number?

-The percent increase questions NPI’s are very difficult to report – most programs have an outcome in terms of number that is much more easily reportable.

-Overall, in all Community NPIs – Percentages should be taken out! This makes it too confusing and does not provide a realistic reportable number.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

-All the information that says auto populated – where is this data coming from?

-Under Infrastructure and Asset Building – could there be another added, specifically, 3. The number of existing assets/resources improved in a specific community. As CAAs, often we are not creating new, but improving on existing.

-Under Health and Social/Behavioral should add three more that would be rather than added… supported. Number of accessible and affordable assets or resources for physical health supported in the specific community. For example, our CAA provides AmeriCorps members to support the operations of the Community Health Clinic.

- Why under Civic Engagement and Involvement are they all percent increase questions… Number would be easier and more realistic to calculate. Then each year you would be able to see the increase through the longitudinal data collected through the report.

Comment 71 (Maine Community Action Association ; Other/Unknown in NE)

Module 3: No Comment

Comment 72 (Massachusetts Community Action Agency Planners’ Group; State Association in MA)

Module 3: · Module 3, Community Level:

Group discussed concerns around specifically defining
“community” for each initiative and how this would impact data collection and reporting. For example, one agency with Head Start may define the community as low-income families eligible for Head Start services, while another may define the community as being all families within the service area; this would completely change the percentage within the “community” achieving the given outcome and make it impossible to compare data across agencies or states. Group also discussed challenge of defining a Collective Impact versus Partnership versus Independent Initiative.

Comment 73 (Maui Economic Opportunity Inc. ; Other/Unknown in HI)

Module 3: No Comment No Comment

Comment 74 (Merced County Community Action Agency ; Other/Unknown in CA)

Module 3: No Comment

Comment 75 (Michigan Community Action ; Eligible Entity/Local Agency in MI)

Module 3: Module 3: Remove the Community Outcome Indicators that include percent or rates. Their inclusion will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or targeted by local agency projects. Many Michigan projects are targeted to the block group level. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable or directly attributable to CSBG. It is more accurate and significant to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

Module 3: Remove Collective Impact from all reports. Collective Impact is one set of strategies for doing community-level work and it has limited use. CSBG eligible entities are engaged in specific initiatives in our state and the way the impact is determined and reported is owned by the members of the initiative. The CSBG Annual Report already has open data fields that can be used to report these impacts. These elements should be removed.

Module 3 - Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

3. Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current "term of are that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options. Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn't yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

1. Remove Collective Impact. See above.

2. See comments later regarding Outcome Indicators

Module 3, Section A: Supplemental Data: (page # 32)

• Remove Collective Impact

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

1. Remove "and percent' from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the "percent" has several problems.

a. First, even though agencies are allowed to define what constitutes "community", it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available).

b. Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility.

c. Third, it may unfairly force a comparison between the agency's outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address "Rates", specifically:

a. Education and Cognitive Development: 5, 6, 7, 10 o Housing: 6, 7, 8
o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, ) o Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

• This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, "agencies must provide a narrative justifying...." is inappropriate.

Comment 76 (Minnesota Community Action Partnership; State Association in MN)

Module 3: Community Level

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

· Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current “term of art” that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

· Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by eligible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone. A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

· Remove Collective Impact. See above.

· See comments later regarding Outcome Indicators

Module 3, Section A: Supplemental Data: (page # 32)

· Remove Collective Impact

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

· A strong case can be made that these proposed data elements would have questionable statistical validity and would almost certainly have very poor statistical reliability. That is to say, it is not at all clear that these data elements would measure what they purport to measure. Furthermore, if 100 people reported on these data elements, there would likely be very little agreement among them with respect to what they thought they had reported. As a result, aggregating these local data at the federal, or even at a state level, has the highly probable effect of knowingly creating misinformation at best and creating disinformation at worst. It is challenging to rationalize how spending scarce CSBG resources to collect this data as proposed achieves a meaningful public good.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

· Remove “and percent” from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

· Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

o Education and Cognitive Development: 5, 6, 7, 10

o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, )

o Public Safety (4a, 4c)

100 Empire Drive | Suite 202 | Saint Paul, MN 55103 | www.MinnCAP.org

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

· This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate

Comment 77 (Minnesota Valley Action Council, Inc. ; Eligible Entity/Local Agency in MN)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Need definition of Partnership vs. Collective Impact. Also how does the network take credit for Collective Impact when we work as part of a larger system? This is not valid reporting, easy to manipulate or make mistakes.

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

By defining target communities from Region to the Neighborhood Level, you are gathering data that cannot (or should not) be aggregated. Impact will be skewed based on size. Agency boards react negatively to initiatives that cherry pick locations vs entire region or service area. Why are we reporting on partners and their roles? These partners likely to not receive CSBG or are part of the network – now we are reporting on outcomes from other agencies.

Module 3, Section A: Supplemental Data: (page # 32)

Durations are too specific – and difficult to choose – if I expect it to take 4 years, do I choose 3-4 or 4-5? The option is repeated.

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

Reporting on number of strategies fails to provide breadth and depth. One strategy may be one-time while another may be ongoing. They are not all equal, so where one agency may report one strategy that takes 6 months, another can report 6 strategies that each take one day. It is impossible to then quantify this impact with any accuracy. All strategies are not created equal!

Module 3, Section B: Community Level NPIs: (page # 37)

Number seems to be appropriate to report. Percent is not – will likely be negligible. Creating 50 jobs is important, but in the grant scheme of my entire region, this is a tiny percent of jobs. You are going to skew away from demonstrating impact. % is a difficult number to assess. Example: It is very subjective on how to report increase of children at the basic reading level in a community – we most likely to not have access to all of those children, however, we can report the number of children who we were able to bring up to a basic reading level.
Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Same comment on number and percent.

I don’t think we want to report the assets we added, but rather the utilization of those assets. That would be a more impactful number. Reporting % of homelessness again is difficult/impossible to report, but reporting # of individuals no longer sleeping on the streets or in shelters is. Likewise, many things can impact foreclosure rates, how do we assess a decrease in foreclosures? Jobs, economy, and other systemic impacts will have as much an impact as a program.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

In Summary, this section of the report will provide neither valid nor reliable data. The impact that you are attempting to measure is subjective, and at times, impossible to measure. Collective impact is not singly our responsibility. Others indicators, especially when reported as a %, can make us appear ineffective as a network as there will be small % change.

Comment 78 (Missouri agency Inc. ; Other/Unknown in MO)

Comment 79 (Missouri Community Action Network (Can); State Association in MI)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section B: Community Level NPIs: (page # 37)

· Employment: The impact that CAAs can have on employment in the community is limited by other factors that community action does not control. Therefore, we do not feel many of these could be reported on effectively.

-Education and Cognitive Development: Many of these sound like individual outcomes. The unit of measurement is individuals/families. A community outcome example: “The percentage increase of the number of post-secondary opportunities available in the specified community.” As cited in the Introduction to ROMA curriculum on page 39 and 40, community outcomes focus on access,
infrastructure, available resources, and opportunities.

· Infrastructure: The first two are appropriate and useful. With indicator 3, it does not seem feasible for community action to put predatory lenders out of business to decrease the number. However, community action could create positive alternatives to predatory lending in the community.

· Housing: “maintenance through WAP” seems very similar to “improving the value or quality through WAP” since the purpose of WAP is to improve quality and energy efficiency. Maintenance through WAP would also improve quality and value.

· Health & Social/Behavioral: #1 is good. #2, there seems to be an indirect relation between service provided and the infant mortality rate. We believe community action has many indirect impacts, but we should only be held accountable for and report on the direct impact. An alternative could be: the increase in opportunities afforded to low-income mothers to ensure healthy babies. #3, b – not sure this is meaningful. Not all unplanned pregnancies are negative and the data may not differentiate.

· Civic Engagement: This is appropriate, meaningful, and useful.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

· Page 38: Housing #4 – percent of what? It makes sense to say 10 shelter beds were established in the specified community. 10% shelter beds cannot be established. The percentage would fit with #5 for the percent increase in shelter beds. Not all need percentages to be meaningful.

· Page 38: #7 – can an agency take the credit for a 10% decrease in the rate of foreclosure with all the factors that might be affecting this decrease? Maybe the agency offers related life skills classes or programs, but other agencies and real estate trends have an impact as well. These rate trends should be examined, but we are unsure this is meaningful information for CAAs to be reporting in the annual report. Moreover, where can this specific information be obtained? In our community commons web data tool (which only 3 states have) we can access foreclosure rates, but even the national CAP community commons web data tool does not include this indicator. Another example is home ownership rates. We can access the number of homeowners, but it is not broken down by income level.

Data should be accessible to agencies if it is an option to be reported. If very few are able to report on it, it is not meaningful to aggregate.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

· Page 41: Education #2 asks for how many children are kindergarten ready in a community. This
information does not seem to be readily available. In a brief search, this was not found in the community commons national or state data tool, the kids count data book, or the Missouri Dept. of
Elementary and Secondary Education data site. If this data is out there, agencies need to know where to find it. If they have to conduct their own research, additional resources and funding will be needed for this level of expertise. Reporting baselines is an option and not all indicators must be used, but if many agencies are not going to be able to find the information, it could be a local indicator rather than a NPI.

· Page 45: Housing #1 – if baseline includes all current safe and affordable housing units (say 100), then in utilizing CSBG and other agency resources it is determined that a realistic target is 2. Therefore, it would be an expected percentage change of 2% from the baseline. If actual results are that 1 home is built, then the actual percentage change is 1% and the performance target accuracy is 50%. Is this useful or able to show the positive impact of community action? If outcomes are being compared to the baseline in the entire community, it may present our impact as minimal and community action will appear ineffective.

Comment 80 (Montgomery County Community Action Development Commission; Other/Unknown in PA)

Module 3: No Comment

Comment 81 (Multi-Service Center (MSC) ; Eligible Entity/Local Agency in WA)

Comment 82 (National Association For State Community Service Program ; Other/Unknown in N/A)

Module 3: 5. Module 3

a. Section A, Numerical Baseline Data: NASCSP believes these sections must undergo additional streamlining. As such, we recommend removing numerical baseline data as a data point on the data entry forms and suggest alternate options detailed later in this letter.

b. Section B, Social/Population Indicators: We recommend categorizing the social/population level indicators as developmental indicators that could be assessed and modified.

c. Social B, Civic Engagement and Community Involvement: The Civic Engagement and Community Involvement goal should be strengthened to further spotlight the involvement of people with low incomes.

Module 3 – Community Level

NASCSP recognizes the complex and systemic causes and conditions of poverty and is dedicated to supporting and advancing the CSBG Network’s effort to address these issues at the individual, family, and community level. As such, NASCSP supports a robust framework for measuring the approaches and impact of community level work across the country. The current proposal in Module 3 expands the number of options and methods of reporting on community level work to ensure data collection will represent the initiatives implemented and outcomes achieved. Specifically, NASCSP supports the Community Level Status page and the data points proposed within that form. We recognize that the Community Level Indicators have not received the same amount of attention in the past several years as the individual and family indicators and are pleased that there is sufficient opportunity during this comment period.

While NASCSP appreciates the importance of a strong framework for data collection on community level work, some aspects of the current proposal are complex and/or beyond the current scope and capacity of the CSBG Network. Specifically, the following items are of concern: Baseline Data for Reporting on National Performance Indicators (NPIs): While NASCSP commends OCS for the great strides taken to tackle the complex issue of tracking community level change, particularly change achieved over time through coalitions or community collaborative efforts, NASCSP recommends that the proposed data entry forms and associated calculations be modified in order to produce meaningful data. NASCSP urges OCS to eliminate some data points before moving forward in the next comment period. The Baseline (i.e. existing starting point used for comparisons) is proposed as an optional data point on the Community Level Initiative Status page and auto-populated on the NPI data entry forms and intended to provide the expected and actual percentage change from baseline. While NASCSP recognizes baseline data is critical to the local process of accurately assessing need, planning for an appropriate initiative, and evaluating efforts, there are concerns associated with including this as a data point as part of the packet moving forward. According to the proposal, the baseline data is optional and will not be aggregated to the State or National level. Consistency of reporting is a concern. In addition, NASCSP questions the value of reporting the baseline at the local level if it will not be used at the State and National levels. NASCSP recommends the following options for modifying this element of the data entry forms: 1) numerical baseline data is not collected in the data entry forms and CAAs instead provide a narrative describing the initiative; or 2) numerical baseline data is not reported in the CSBG Annual Report and baseline data remains a training and technical assistance effort at the local level and is included in the Community Needs Assessment to inform the Community Action Plan and approach for the year.

Community Level National Performance Indicators: NASCSP is keenly aware of the vast diversity of approaches implemented by local agencies to address community needs. The menu of options proposed in the CSBG Annual Report provides CAAs with a wide range of indicators in order to demonstrate community level initiatives. In addition to all of the optional indicators, we support the addition of the “other” indicator option as especially helpful in ensuring agencies are able to capture unique community outcomes. Nonetheless, the population/social indicators included in the proposal are extremely complex, difficult to measure and track, and may not accurately represent the efforts of local CAAs. NASCSP understands the concerns around social/population indicators that were expressed during the previous feedback period, as well as the fact that there are CAAs that are working toward these outcomes. Given the feedback from.

NASCSP Feedback – FRN #1 OMB PRA Clearance, CSBG Annual Report the CSBG Network and the diverse approaches to community level work, NASCSP recommends keeping the social indicators in the CSBG Annual Report, but classifying the population/social indicators as developmental indicators. As a developmental indicator, this would allow for the CSBG Network to assess the clarity and utility of the indicators in capturing community level work. This revision will acknowledge the vast range of CAAs’ involvement in community level initiatives, as well as the fact that needed changes within communities take a great deal of time, investment, commitment, and resources, and are often dependent on other factors outside of the scope of agencies and their partners.

Emphasis on Civic Engagement and Community Involvement: NASCSP believes it is critical that local agencies analyze community needs and conditions and aggressively pursue partnerships that address the causes and conditions of poverty. As such, the additional data points on the Community Level Initiative Status page related to partnerships are valuable in realistically telling the story of community level work. NASCSP recommends including additional emphasis on the involvement of people with low-incomes (i.e. stakeholders) in informing and contributing to community level work. We believe this will speak to CSBG’s core principle of maximizing involvement of people with low incomes, ensuring initiatives are effective by including all appropriate stakeholders, and add additional context to proposed goal 3 “People with low incomes are engaged and active in building opportunities in their communities”. One possible option for addressing this issue is to include stakeholder involvement in the Community Level Initiative Status page.

Comment 83 (National Community Action Foundation ; Eligible Entity/Local Agency in N/A)

Module 3: Module 3- the Community Impact Reports: Most of the forms in this portion of the report should be designated as voluntary, not required, tools for local use.

National Community Action Foundation ROMA NG Final Comment

1. Intrusive detail required has little value for local management nor for state and federal oversight roles.

The draft report is far too detailed and requires data collection and analysis that are irrelevant to the outcomes CAAs seek to achieve. While the proposed report forms may be sound project management tools for some types of community projects, they are inappropriate as federal reports.

In April comments to NASCSP on the first draft ROMA NG, we offered a simplified format that captured the goals local CAAs’ community programs sought to achieve. Our proposed alternative records quantitative outcomes for as long as the project generates value, and includes an indicator of whether or not the initiatives are meeting their planned goals. We believe any other information is unnecessary to federal oversight.

States will be in the position to verify the progress and nature of community projects through their monitoring and may choose to supply narrative "stories" as in the current report system, which requires further paper work. Furthermore, CSBG is rarely the dominant funder of investment in community capital. Even small-scale projects typically leverage multiple sources, each of which demands a particular reporting form. Given this fact, the proposed reports layer an additional paperwork burden on projects, but are collecting tracking data, not results.

2. The “partnership characteristics” portion is a poor fit to CAA partnership and evidence-based good practices.

The checklist of partnership characteristics can be useful for writing standardized, local descriptions of projects. The problem is with the detailed narrative reports that are not needed to measure the CSBG’s success, which again, should be the foundation of any CSBG reporting system.

National Community Action Foundation ROMA NG Final Comment

Further, the mandated template includes terms used in the fashionable “Collective Impact” theory that are inappropriate ways to characterize projects. The proposed system’s Collective Impact foundation will be significantly less useful than alternative frameworks over the next decade that the ROMA system will be in place.

Collective Impact framework was published by the principals of a consulting firm, FSC, and featured in the Stanford Innovation Review in 2011. It has simple principles for operations and a limited number of variants for the structure of effective collaborations, all of which involve one lead agency they labeled a “Backbone” organization. The Backbone has a defined set of roles, and a budget of at least $100,000 for the work. In the years following publication, the authors elaborated on the concept, and numerous philanthropies requested and funded proposals structured according to the principles. However, there is little evidence basis for the framework. The authors rely upon a small set of case studies, and, post facto, characterize the roles and relationships in the effective partnerships.

On the other hand, there is ample evidence base for the success of collaborations that do not involve a dominant partner, especially in disadvantaged communities or any community where inclusion or community trust have been at issue. In the piece, “Ten Places where Collective Impact Gets It Wrong,” published by the Global Journal of Community Psychology Practice (Vol. 7, Issue 1, March 2016), respected community psychology scholar, Tom Wolff, reviews the substantial scientific literature regarding the successful elements of both service and policy collaborations. In most cases, successful collaborations have very different structures than the Collective Impact model. Nothing in the reporting framework should bias CAA project design or partnerships to one organizational theory.
Recommendation: All the specialized Collective Impact terminology and categories should be dropped in favor of plain English in order to promulgate voluntary partnerships descriptive tool that has enduring validity when utilized by local agencies and their partners. NCAF submitted an example to the CSBG working group and to NASCSP earlier this year.

NCAF and our national partners at the Community Action Partnership, along with many community action leaders, have responded to OCS that community demographic indicators, which most agencies consistently use in their process of assessing community needs, are not appropriate outcome measures for their community projects. We urge OCS to replace demographic impact measures with the actual goals of the projects.
Our alternative: Like the community strategies list, which is a thoughtful and welcome reference guide, OCS can provide sample indicators of impact that are appropriate for measuring community impacts and that are not the equivalent of totaling up a large number of individual outcomes. For example, OCS could draw a distinction such as this:

• “Forty families developed better reading-to-children habits” is not a community impact.

• However, “the school pyramid adopted the parent support model of the ‘Reading is a Habit’ initiative. The initiative trained classroom aides as parent coaches. Because the test in 4 schools changed, the habits of 80 percent of participating parents changed” is a community impact statement.

Why are the proposed demographics indicators inappropriate?
The scale of the available information is wrong. Census and economic data are valid for different, usually larger geographic areas than the scope or target of CAA projects. Small-scale, multi-year, cross-boundary projects are typical Community Action change undertakings. Many are demonstrations later adopted and funded by an entire jurisdiction or other organization. Some are campaigns.

1. The data are not available for the combination of neighborhoods in many projects, nor the specific populations addressed.
National Community Action Foundation ROMA NG Final Comment
2. The effect of external variables, such as population flow and the state of the economy, on the population, measured by the draft indicators, is likely greater than the impact of the project.

3. The demographics measures do not reflect the goals of many past or potential initiatives.

For example:

A CAA may operate a project to train school aides and counselors to recognize and respond to the causes and symptoms of stress in children living in poverty. In such a situation, the program will facilitate appropriate parent-teacher contacts by training and coaching both parties. Outcomes should be measured by:

I. The institutional effects within the school;
II. Observations on parents and perhaps other children not included in the original project; and,

III. The effect of a successful demonstration on other schools in the school department.

All of those measurements rely on data unique to the school population and catchment area.

4. The proposed measures represent HHS’ definition of success, not the objectives of the carefully selected local initiatives.

5. Such demographics have no meaning when reported as laid out in the Module 4 framework. To have meaning, such demographics would need to be compared to a control group or area. This would be the case even if a statistic reflected the precise scale and objective of a CAA project.
For instance, employment indicator #5, as proposed in the report, cannot determine the success of a project aimed at bringing an employer, with well-paying jobs and benefits, to a community. The indicator, defined as “the number of jobs in the targeted community with a benefit package,” would not measure the effect of the CAA project. That indicator measures the locality’s workforce with benefits. Perhaps it can measure change over time, but it does not provide for any comparative indicator over time or contrast to a market where the CAA was not involved.

There are two reasons the indicator cannot demonstrate the impact of a CAA project. First, the CAA project may not be large enough to generate noticeable statistical changes to the percent of benefit-paying jobs in an economic census of a large geographical area. Secondly, a project might look successful because of factors external to the CAA project, such as a state law requiring paid leave or implementing the Affordable Care Act regulations that incentivize employer-provided health benefits. All socio-economic indicators are only informative compared to an equivalent in a control group.

Because of these reasons, collecting and reporting on the proposed demographic measures has no value for outcome management.

Comment 84 (National Community Partnership ; Other/Unknown in ND)

Module 3:

Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

· Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current “term of art” that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

· Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

· A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

· Remove “and percent” from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

· Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

o Education and Cognitive Development: 5, 6, 7, 10

o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, )
o Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

· This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate.

Comment 85 (Nek-Cap, Inc. (Northeast Kansas Community Action Agency); Eligible Entity/Local Agency in KS)

Module 3:

Community level initiatives ultimately produce the greatest outcomes by creating producing systemic change through community alignment in addressing the causes and conditions of poverty and promoting economic opportunity. Community level initiatives require dedicated resources of time, staff, and funding for a minimum of 3 to 5 years and must have a concurrent sustainability plan to produce both short term and long term outcomes. The Community Level Module 3 data sets and strategies are currently not collected through the CAP 60 software system and have not been the agency’s primary focus through CSBG funds. Community level initiatives would require the agency to redirect the use of such funds and services and would also necessitate additional funding to support such initiatives.

Comment 86 (New England Community Action Partnership (NECAP) ; State Association in CT)

Module 3: Module 3 – Community Level: (pages 28-50)

· Module 3, Section A: Community Level Initiatives Home Page: (page # 29) o Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently.

O Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

o A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

· Module 3, Section B: Community Level NPI Landing Page: (page # 37-39) o Remove “and percent” from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator.

O Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, especially Education and Cognitive Development: 5, 6, 7, 10; Housing: 6, 7, 8; Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c,); and, Public Safety (4a, 4c).

· Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50) o This instruction is misleading. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is out of place.

Comment 87 (New York Department Of Youth And Community Development ; State in NY)

Module 3:

1. It is unclear how HHS would aggregate useful information related to Community Outcome Indicators that include percentages or rates, when these presumably would be used to measure very different things from one Community Action Agency to another

Comment 88 (New York State Community Action Association; Eligible Entity/Local Agency in NY)

Module 3: Module 3 – Community Level: (pages 28-50)

· Collective impact is a set of strategies and term of art currently used to describe efforts to engage in community level work but is neither the only nor preferred option and should be removed from the reporting tool.

· Social or population indicators are unnecessary and provide little useful information on the impact of community level initiatives and strategies. As noted above, reporting and evaluating the success of community-level interventions requires a rigorous evaluation that accounts for the influence of external factors and allows for the long time period in which community-level indicators are undertaken. An effective and useful assessment should include the ability to report over the long time frame within which the intervention occurs, the types of strategies employed, the target population and the outputs achieved during the long process on implementing a truly effective community-level initiative.

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)
· As noted above, it is recommended that the Collective Impact strategy be removed.

Module 3, Section A: Supplemental Data: (page # 32)

· Remove Collective Impact
Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

· Including the percent in this section is highly misleading and may result in unduly negative assessments of the impact of a community level intervention. It is difficult to identify a “community” and find corresponding data, for example for a community education project. Furthermore the percentage may be misleading, for example creating 25 day care slots in a community with a high unmet need for these slots may not look like a successful undertaking when in fact it achieved the realistically goal set. .

Comment 89 (Ninth District Opportunity, Inc.; Eligible Entity/Local Agency in GA)

Community Level:

The report is a big improvement on the original report. The data for this report could easily be integrated into a client tracking system. As has been mentioned, this entirety of the report (but especially this section) should be able to be digitally transferred.

Module 3, Section A: Community Level Strategies

Domains should be numbered an in alignment with CSBG expenditure categories. Every service category should have a direct link to a CSBG expenditure category, including community support and emergency management. If it is not feasible to add an expenditure category, services under these categories should be reassigned so that a direct link can be made. Each strategy should be provided with a unique identifier for consistent tracking purposes.

Module 3, Section B: Community Level NPIs

Domains should be numbered an in alignment with CSBG expenditure categories. Every NPI category should have a direct link to a CSBG expenditure category/service category.

Every service/strategy should be able to be linked to an outcome. While it is understood that the goal of the network is to focus on long-term change, outcomes should exist that coincide with the strategies listed under Emergency Management.

Each NPI needs to be provided with a unique identifier for consistent tracking purposes.

Comment 90 (North Central Community Action Program; Eligible Entity/Local Agency in WI)

Module 3: -I don’t see how we will be able to measure “collective impact” in any meaningful way.

-I don’t know how we will rate or give percentages on many of the areas proposed. The criteria for determining a percentage is not clear and could be misleading.

-The definition of a “community” seems very ambiguous and isn’t something that can be adequately measured. I am not sure how we would collect some of this dat.

Comment 91 (North Dakota Community Action ; State Association in ND)

Module 3: Module 3 — Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current "term of art" that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal
agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

A more appropriate system may allow a local agency to report (I) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn't yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

• Remove Collective Impact. See above.

• See comments later regarding Outcome Indicators.

Module 3, Section A: Supplemental Data: (page # 32)

• Remove Collective Impact.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

• Remove "and percent" from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the "percent" has several problems. First, even though agencies are allowed to define what constitutes "community", it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency's outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address "Rates", specifically:
Education and Cognitive Development: 5, 6, 7, 10

Housing: 6, 7, and 8

Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c,)

Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

e This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, "agencies must provide a narrative justifying....' is inappropriate.

Comment 92 (North Dakota Department Of Commerce; State in ND)

No Comment

Comment 93 (North Dakota Lead Agency; State in ND)

No Module 3 Content for this comment.

Comment 94 (Northeast Kansas Community Action Agency; Eligible Entity/Local Agency in KS)

Module 3: Module 3 – Community Level:

Community level initiatives ultimately produce the greatest outcomes by creating producing systemic change through community alignment in addressing the causes and conditions of poverty and promoting economic opportunity. Community level initiatives require dedicated resources of time, staff, and funding for a minimum of 3 to 5 years and must have a concurrent sustainability plan to produce both short term and long term outcomes. The Community Level Module 3 data sets and strategies are currently not collected through the CAP 60 software system and have not been the agency’s primary focus through CSBG funds. Community level initiatives would require the agency to redirect the use of such funds and services and would also necessitate additional funding to support such initiatives.

Comment 96 (Northern Tier Community Action Corp. ; Eligible Entity/Local Agency in PA)

Module 3: Module 3, Section B: Community level NPls; (page # 37)

Why report on the percentages on these indicators? This is confusing.

Comment 98 (NORWESCAP; Eligible Entity/Local Agency in NJ)

Module 3: No Comment

Comment 99 (Northeast Community Action Agency ; Other/Unknown in FL)

Module 3: The Community Level data collection tool seems to be an excellent tool for Planning and reporting. I feel that it defines and sets the standards for community level reporting. It requires agencies to validate stated results. However, some community data collection will be burdensome or just not available in some communities.

Module 3, Section A: Community Level Initiatives Home Page: Page #29

Independent level community initiative is questionable and should be deleted. Collective community initiative should be the standard.
Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

No comment

Module 3, Section A: Supplemental Data: (page # 32)

No comment
Module 3, Section A: Strategies Popup Windows: (page # 33-36)

Excellent tool

Module 3, Section B: Community Level NPIs: (page # 37)

No comment

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Living wage outcomes should not be included due to living wage not being defined consistently. Some data collection to validate outcomes might be burdensome and may discourage some agencies from attempting to achieve those outcomes.

Education and Cognitive Development: Some of the outcomes listed seem to be out of the scope of community action agencies. This section should be reviewed and some outcomes eliminated from the NPI. For example:

* Percent increase of children in the community who are kindergarten ready in the specified community.

* Percent increase of children at (or above) the basic reading level in the specified community.

* Percent increase of children at (or above) the basic math level in the specified community.

* Percent increase in high school (or high school equivalency) graduation rate in the specified community.

* Percent increase of the rate of youth with low incomes who attend post-secondary education in the specified community.

* Percent increase of the rate of youth with low incomes who graduate from post-secondary education in the specified community.

* Percent increase in the number of adults who attend post-secondary education in the specified community.

The number (and percent) increase of adults with low incomes who graduate from post-secondary education in the specified community.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

Comment 100 (Nothing Compares; Other/Unknown in NC)

Module 3: Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

All collective impact options should be removed from the Annual Report. Specific to North Carolina, less than .056% of eligible entities indicate having an understanding of collective impact. It may be beneficial to offer collective impact as a service model at the national and/ or state levels or possibly highlight finding resources available to pilot and research these efforts;
however, we strongly believe this should not be a part of annual national report.

All baseline data points should be removed from the Annual Report. Having baseline as a data point may jeopardize eligible entities ability to report performance data as this information may not be readily available. Further, information that cannot be aggregated at the national level
should not be included in a national report.

Module 3, Section B: Community Level NPIs: (page # 37)

Remove indicators that track increase or decrease in rates or percentages. Reporting changes in rates of social or population indicators is inappropriate and will not provide clarity to the impact of community-level initiatives and strategies implemented by eligible entities and their partners. Additionally, these indicators are flawed as presented due to tracking change in only a single direction (increase or decrease) vs. movement in either direction and they inappropriately place
the accountability on the single agency reporting when there are multiple driving factors in changes to such rates. Again, information that cannot be aggregated at the national level should not be included in a national report.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

See Response Above

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

See Response Above

In addition, remove I. Target Community; Il. Baseline, IV. Expected Change from Baseline and VI. Actual Change from Baseline data points for collection.

Items IV Expected Change from Baseline and VI Actual Change from Baseline cannot be aggregated. Below is an example of the risks this current structure presents:

Employment -

CREATION

1. The number of jobs created to increase opportunities for people with low incomes.

I.) Target

Community

(Specify city, county, etc.) City

Il.) Baseline (existing

IV.) Expected % change from V.)

m.)

starting point

baseline

used for comparisons) 2000

Target
50

(Target) (% auto calculated) 2.5%

Actual

(results)

50

VI.) Actual % change from baseline (% auto calculated) 2.5%

Performance target accuracy (% auto calculated) 100%

There are 2000 jobs for baseline when targeting. Assume a factory closed resulting in the loss of
150 jobs. If the local Community Action Agency created 50, that would put the community at 1900 (2000-150+50) or at a -5% change vs the +2.5% being presented. The assumption made based on the presentation of data elements above will be this city experienced a 2.5% increase in
job opportunities for low-income people. However, we cannot make the assumption the 2000 baseline was maintained. Nor is it reasonable to expect agencies to track total jobs in the defined community per year and take credit of all increase or decrease. Further, it is unreasonable to assume that job creation specifically targeted opportunities for people with low-income. What constitutes jobs for people with low-income vs. any job created? Again, information that cannot be aggregated at the national level should not be included in a national report.

Comment 101 (Ohio Association Of Community Action Agencies; Other/Unknown in OH)

Module 3: No Comment

Comment 102 (Opportunities For Otsego, Inc. ; Other/Unknown in NY)

Module 3: Module 3, Section A: Community Level Initiatives Home Page:

· Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current “term of art” that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

· Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

· A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section B: Community Level NPI Landing Page:

· Remove “and percent” from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

· Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

o Education and Cognitive Development: 5, 6, 7, 10 o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, ) o Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms:

· This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate.

Comment 103 (Oregon Housing And Community Services ; Other/Unknown in OR)

Module 3: No Comment

Comment 104 (People Incorporated Of Virginia; Other/Unknown in VA)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current “term of art” that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

Remove all rates and percentages, where numbers are required, they should be absolute numbers. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact (see the employment example in Question 3 at the beginning of this letter), meaning the data is of limited utility and open to damaging misinterpretation. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).
Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

We strongly recommend removing Collective Impact; see above.

Module 3, Section A: Supplemental Data: (page # 32)

Remove Collective Impact.

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

No comment.

Module 3, Section B: Community Level NPIs: (page # 37)

No comment.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Remove "and percent" from all the community indicators. OCS has stated clearly that agencies would define their own communities so as to determine percent. This will render data aggregation meaningless at the state and federal levels.

Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. As noted above, these elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

o Education and Cognitive Development: 5, 6, 7, 10

o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, )

o Public Safety (4a, 4c)
Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local agencies have the responsibility to determine which strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate, particularly in regards to a block grant-funded program.

Comment 105 (ProAction Of Steuben And Yates Inc.; Eligible Entity/Local Agency in NY)

Module 3: Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

· Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current “term of art” that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

· Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

· A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

· Remove Collective Impact. See above.

· See comments later regarding Outcome Indicators

Module 3, Section A: Supplemental Data: (page # 32)

· Remove Collective Impact

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39) Remove “and percent” from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

· Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

o Education and Cognitive Development: 5, 6, 7, 10

o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, )
o Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

· This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate.

Comment 106 (Project Bravo; Eligible Entity/Local Agency in TX)

Module 3: Finally, it seems like many of the concepts proposed for the new reports favor a collaborative approach for CAPs within each state. I think that Texas has done a great job in working together to comply with federal CSBG requirements, but there are limits to our capabilities. Our 40+ CAP programs cover 269,000 square miles of territory. I am lucky that I only cover about 1,000 of those 269,000 square miles, but it is not easy. I have colleagues that have dozens of counties and ten times the service area. We rely heavily on teleconferences in order to collaborate, plan, and train on CSBG initiatives. We travel to meet as often as we can, but that usually means people drive for half a day or fly which is very expensive (I usually fly because it would take me 9 hours to get to Austin, TX). I suspect that during our teleconferences many agencies can’t speak up or bring up concerns because there isn’t enough time to ensure all 40+ voices are heard. I can’t imagine how we are going to juggle the needs of 40+ CAPs helping 4.5 million people in poverty while we work to comply with the new model

Comment 107 (Pueblo County Housing And Human Services; Other/Unknown in CO)

Module 3: Module 3: Community Level / No Comments

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

3. The quality, utility, and clarity of the information to be collected; and

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Comment 108 (Redwood Community Action Agency ; Eligible Entity/Local Agency in CA)

Module 3: Module 3 — Community Level: (pages 28-50)

Our organization supports the importance of collecting information about the work CAAs do impacting community level change over time periods greater than one year. However, we understand that not every CAA is engaged in this breadth of activity and/or may not be able to fully understand how to report on this type of activity. Therefore, while this might be applicable data collection for some, as a whole the information may not have practical utility at the State and National Level.

Furthermore, the requirement for community level Baseline Data which would be used to assess Actual Results is: (1) in many cases not available for the community size for most indicators (especially in rural areas); and (2) will not have practical utility as it cannot be used for comparison or analysis at the State or National level due to the range in size and populations of a CAAs self-defined "community." The requirement to provide this baseline data and then report on outcomes in this way does not seem either practical or useful; the option to provide narrative rather than data further shows that information gathered in this format will not be able to show the impact of this community level work to any great extent. The burden of collection and reporting data in this module as presented far outweighs the utility of any data gathered.

Comment 109 (Sa Howell, LLC; Other/Unknown in GA)

Module 3: Module 3
This report is a big improvement on the original report. The data could be easily integrated into a client tracking system. As has been mentioned, the entirety of the report (but especially this section) should be able to be digitally transferred.
Module 3, Section A

Domains should be numbered and in perfect alignment with expenditure categories. Every service category should have a direct link to a CSBG expenditure category, including community support and emergency management. It is not feasible to add an expenditure category, services under the category should be reassigned to a direct link can be made.

Each strategy should be provided with a unique identifier for consistent tracking purposes.

It is not necessary to specify the specific community strategy on a national level. They can be generalized (employment initiative, education initiative, etc.) and specified on a local or state level.

Baseline rates and percentages will be an undue hardship on agency planners, and demonstrates no clarity to the impact of community level initiatives.

Module 3, Section B

Domains should be numbered and in alignment with CSBG expenditure categories. Every NPI category should have a direct link to a CSBG expenditure category and service category. This connection MUST be able to be easily made for planning and evaluation purposes. This is not optional.

Every service/strategy should be able to be linked to an outcome. While it is understood that the goal of the network is to focus on long-term change, short-term outcomes should exist that coincide with strategies listed under emergency management.

Each NPI needs to be provided with a unique number for consistent tracking purposes!

Remove “and percent” from all community level indicators.

Comment 110 (SEMCAC; Eligible Entity/Local Agency in MN)

Section A: Community Level Initiatives Home Page: (page # 29)

· Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current “term of art” that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

· Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

· A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

· Remove Collective Impact. See above.

· See comments later regarding Outcome Indicators

Module 3, Section A: Supplemental Data: (page # 32)

· Remove Collective Impact

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

· No Comments

Module 3, Section B: Community Level NPIs: (page # 37)
· No Comments

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

· Remove “and percent” from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the “percent” has several problems. First, even though agencies are allowed to define what constitutes “community”, it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

· Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address “Rates”, specifically:

o Education and Cognitive Development: 5, 6, 7, 10

o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, )

o Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

· This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate. When agencies are attempting to report on the numerous community indicators, there is not a list of what to reference as resources to produce comprehensive reports from agencies, particularly when the data is combined on the state level/national level. The current state of these reports allow for agencies to simply utilize any outside resource (national, community, state, institution study, even politically fueled resources, etc.) to compare the data and create the “baseline” which the remainder of the report will be based upon. This produces incomparable results when gathering overall network data and thus is not a clear reflection of what CAP agencies have accomplished for the communities they serve. If these forms are going to be left within the report, there needs to be some sort of cohesive resources for us to utilize.

Comment 111 (South Plain Community Action Association ; State Association in TX)

Module 3: No Comment

Comment 112 (Southeastern North Dakota ; Eligible Entity/Local Agency in ND)

Module 3: Module 3 — Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of the agency. It will have no utility at the state and national levels as described currently. Collective Impact is a current "term of art" that refers to a specific set of strategies for engaging in community-level work and it makes little sense to privilege this approach over all other options.

Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal
agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by legible entities and their partners. These elements are used in community scorecards and community needs assessments, not as an outcome indicator for an agency to a state CSBG office. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

A more appropriate system may allow a local agency to report (1) the overall domain of the community initiatives (e.g. health, workforce, etc.); (2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); (3) the target population (e.g. all at-risk high school students in a three county area); and (4) Outputs or Counts (this would allow for agencies to get credit for activities that move the initiative forward but hadn't yet resulted in outcomes).

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

• Remove Collective Impact. See above.

• See comments later regarding Outcome Indicators.

Module 3, Section A: Supplemental Data: (page # 32)

• Remove Collective Impact.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

Remove "and percent" from ALL community level indicators. Using percent is confusing and negatively impacts the quality, utility and clarity of the data and will lead to data misuse and misinterpretation. Including the "percent" has several problems. First, even though agencies are allowed to define what constitutes "community", it may be difficult for agencies to collect this data if it does not correspond to a unit or level of analysis for which there is readily available data (e.g. if community is defined as several census tracts for which the specific data in question is not available). Second, because the definition of what constitutes the community is left up to the agency, agencies may struggle to define it exactly (e.g. in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Third, it may unfairly force a comparison between the agency's outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look like (misleadingly) like a minimal result when compared with the overall county unemployment rate.

Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used in community score cards and collected as part of a needs assessment. Remove all indicators that address "Rates", specifically:

Education and Cognitive Development: 5, 6, 7, 10

Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c, )

o Public Safety (4a, 4c)

o Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

This instruction is misleading and is inappropriate. It will negatively impact the capacity of the agency to perform its functions properly. Local CAAs have the ability to determine the strategies they implement in a community and to have the reporting tool state, "agencies must provide a narrative justifying....' is inappropriate.

Comment 113 (Southern New Hampshire Services ; Eligible Entity/Local Agency in NH)

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

The use of percentages and rates seems very confusing, and is likely to produce useless information. An agency may well look at these statistics on a local level as part of a community assessment or planning initiative, but using percentages in aggregating data at a state or local level leaves open the possibility of misinterpretation. Graduation rates or literacy rates, for example, can be impacted by, but are not determined by CAAs alone. This brings into question their usefulness as measures of the impact of a local agency.

Additionally, even though CAAs define the community they serve, that does not automatically translate into being able to get accurate data sets for various metric from that community, e.g. certain census tract blocks, or other geographic areas where data may be spotty or unavailable.

Comment 114 (Southwest Oklahoma Community Action ; Other/Unknown in OK)

Module 3: No Comment

Comment 115 (State Of North Carolina; State in NC)

Module 3: Module 3 Community Level

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

All collective impact options should be removed from the Annual Report. Specific to North Carolina, less than .056% of eligible entities indicate having an understanding of collective impact. It may be beneficial to offer collective impact as a service model at the national and/ or state levels or possibly highlight finding resources available to pilot and research these efforts; however, we strongly believe this should not be a part of annual national report.

All baseline data points should be removed from the Annual Report. Having baseline as a data point may jeopardize eligible entities ability to report performance data as this information may not be readily available. Further, information that cannot be aggregated at the national level
should not be included in a national report.

Module 3, Section B: Community Level NPIs: (page # 37)

Remove indicators that track increase or decrease in rates or percentages. Reporting changes in rates of social or population indicators is inappropriate and will not provide clarity to the impact of community-level initiatives and strategies implemented by eligible entities and their partners. Additionally, these indicators are flawed as presented due to tracking change in only a single direction (increase or decrease) vs. movement in either direction and they inappropriately place
the accountability on the single agency reporting when there are multiple driving factors in changes to such rates. Again, information that cannot be aggregated at the national level should not be included in a national report.

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39) See Response Above

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

See Response Above

Comment 116 (Texas Association Of Community Action Agencies, Inc. (TACAA); Other/Unknown in TX)

Module 3: No Comment

Comment 117 (Texas Department Of Housing And Community Affairs ; State in TX)

Module 3: Over the past few years, our Department has put great effort and resources in providing the Texas CSBG network with basic ROMA training. In the last two years, the network has diligently addressed the task of improving and/or creating systems to ensure adherence to the CSBG Organizational Standards and to minimize noncompliance. The State is still continuing to focus our training and technical assistance in these two areas and in other areas of need. Implementation of ROMA Next Generation, along with the new CSBG Annual Report, will be a big undertaking that will require many resources at the local, state, and national level.

In closing, assessment of the design and structure of the proposed reporting of performance for community action agencies must include a critical evaluation of the volume of data requested and the practical implementation and proposed results of the proposed changes to the CSBG Annual Report. And, a realistic assessment of whether the new Report will show that the CSBG Network makes a difference in the lives of the persons it serves or whether it will appear that very little is being done in the innumerable areas for which we are collecting data.

MODULE 3, SECTION A:

1. Section A needs instructions that are clear and concise and simple as to what is to be reported.

The term, “Initiative” needs to be defined with specific examples. We recommend providing examples of different types of community level initiatives. The area of community initiatives currently has low reporting for Texas. Very few of our subrecipients have the funding to support community projects. We anticipate this is the same for other states. Therefore, consider simplifying and minimizing what is to be collected.

2. We recommend not using the term Collective Impact Initiative. It may be hard to differentiate between an initiatives that is part of a partnership versus a Collective Impact Initiative. We are not clear what the relevance is of the various choices.

3. There are too many questions that complicates the reporting, causes confusion, and detracts from the more significant and notable performance. It does not appear significant or necessary to have: Expected Duration, Initiative Year and Baseline (which may be difficult to estimate). We recommend simplifying the “Strategy (ies)” and not have the huge drop down menu with 9 categories and 95 strategy choices. Remove all strategy choices and replace with the 9 categories and have the sub recipient briefly describe the strategy. Another option is to have 3 broad choices plus an “Other” choice under each of the categories.

4. Perhaps consider changing this section to a very simple and brief form which asks:

Describe the initiative.

Is it a new or ongoing initiative? (they’d check a box)

What is the anticipated duration?

What outcome is desired?

How many persons are anticipated to be served each year once the initiative is operational?

MODULE 3, SECTION B

Overall comments MODULE 3, SECTION B: Too much information is being collected in this section. Prior comments included comments which stated that some NPIs were not reflective of CAA work and beyond the scope of CAA, it is hard to show the correlation between improvements and CAA work, etc. It appears that this comment was not adequately considered since the same still holds true in the measures being proposed. The comment that this section be streamlined was also not considered. This section is still very complex and collects too much information (areas within domains). We also think that it is not necessary that every Individual and Family Level outcome also have a related Community Level NPI.

We recommend that Data Entry Forms for Module 3, Section B be revised to remove II. Baseline and IV. Expected % change from baseline. We recommend the use of similar columns as Module 4 Data Entry Form. The collection of baseline information will be time intensive and require a certain level of research skill and more importantly, when the actual impact is measured based on reported performance, we anticipate that the impact will be very small/limited and will not help illustrate the good work that the CSBG network does, but the focus of observers will be on how small the impact is.

Also, there is a lot of overlap between Module 4 Individual and Family Level measures and Community Level Measures. Module 4 captures the persons and Module 3 captures number or percent increase in the outcomes. Our recommendation is to streamline Module 3 to the bare minimum or merging Module 3 and 4 and eliminating duplicate categories from Module 3. Or, consider merging all of Module 3 Section B with Module 4 Section C. We recommend keeping the current Community Improvement and Revitalization and Community Quality of Life and Assets report sections instead of creating a whole new Module 3. The old measures were broad enough to encompass quite a bit of activity. Adding more guidance and examples to the guide would be helpful.

Lastly, we recommend conducting a conference call the top 12 CSBG funded states and inquire as to their expectation of the numbers they anticipate receiving with these new measures.

MODULE 3, SECTION B:

Employment

1. For Item #1 recommend the following revision. Delete “and percent” and rewrite to read “The number of jobs created.” The revised text is recommended because 1) the percent may be difficult to determine and will likely be very small. Revising to capture jobs created has a greater impact. Also, the jobs aren’t necessarily created for low-income people only. When you add that qualifier, it will be even less jobs reported.

2. For items #1-4 delete “in the specified community” and leave it open to the service area. And for #5 take out “in the targeted community. Agencies will just report for their service area.

MODULE 3, SECTION B

3. For items #1-#6 recommend deleting “percent” due to increased staff and time necessary to determine a percentage of the population. Also, in most cases, the percent will be negligible to report and it won’t show much of an impact.

4. Recommend merging #1 and #2 to read “The number of jobs created or maintained.

Recommend merging #3-#4 to read “The number of living wage jobs created or maintained.” Or, consider merging #1 thru #4 to into one measure to show more impact. Delete #5 or perhaps be merged with previous measures.

Education and Cognitive Development

1. Item #1 Please define an education asset. It’d be best to use the terms like education “buildings or services” instead. Also, it refers to “affordable education assets.” Affordable needs to be defined. There is much room for interpretation of “affordable,” plus this limits what can be reported.
2. For items #2-#8 and #10, recommend removing percent increase and instead have them report

“The increase in the number of children who are….” Also, delete the terms “in the specified community” and just report activity in the CSBG service area. While we understand the intent is to show change, it is likely that the data reported will have very low numbers. Requesting that sub recipients determine a percent increase for their community is a reporting burden, a high probability that sub recipients will have difficulty determining the percent increase because they have to determine the whole population that is available.

Infrastructure and Asset Building

1. For items #1-#3 recommend not requesting percent. It is not clear whether by requesting a percentage, that it would be expected that sub recipient know how many of those resources there are in the specified community. Or is the percentage increase referring to the assets that the sub recipients would not be able to necessarily increase? While we understand the intent is to show change, it is likely that the data reported will have very low numbers. Requesting that sub recipients determine a percent increase for their community is a reporting burden, a high probability that sub recipients will have difficulty determining the percent increase because they have to determine the whole population that is available.

Housing

1. For items #1-#8 recommend not requesting percentage from baseline. It is not clear whether it would be expected that the sub recipients know how many resources there are in the specified community. Is this referring to percent increase in the assets that our sub recipient has control over? While we understand the intent is to show change, it is likely that the data reported will have very low numbers. Requesting that sub recipients determine a percent increase for their community is a reporting burden, a high probability that sub recipients will have difficulty determining the percent increase because they have to determine the whole population that is available.

MODULE 3, SECTION B
2. One-third of the measures are measures that the Network is not likely to report on, specifically

#4 thru #6. Also, some of what is being measured is not very likely to see increases, like #4 and #5 shelter beds. The movement at the federal and state level is to decrease the number of shelter beds and to rapidly re-house persons. Measure #6, decrease in rate of homelessness, it is not likely that a community action agency can attribute their activity to this decrease. Also, how would these numbers be aggregated at a state or national level?

3. Measure #7 percentage decrease in foreclosure rate, again, it is not very likely that a community action agency can attribute their activity to this decrease. We recommend deleting this measure. Also, how would these numbers be aggregated at a state or national level?

4. Recommend only keeping #1 thru #3 and merging with Module 4 perhaps the community level measures could be itemized first and then the family and individual ones under each category.

Health and Social/Behavioral

1. For #1 a.-c. (Please refer to pages 46 of 75 of FRN#1 CSBG Annual Report merge a. and b. because it is likely that very low numbers will be reported in these measures.) For #1.c. on food options, we anticipate a very low percent of the network reporting in this area. With the qualifiers of “healthy, accessible, and affordable food options” it narrows the reporting outcomes. If #1.c. is maintained, recommend deleting some of the specific language “healthy” “accessible.”

2. For #2. a. mortality rate, b. childhood obesity rate, and c. adult obesity rate, delete these measures. This is not a focus of the network and reporting would likely be extremely low. Moreover, it would be difficult to support that their efforts (alone or in partnership) was what led the decrease. Also, the baseline II. may be difficult to obtain and the IV. Expected change will be low. Recommend only keeping measures 2.d. immunization and measure 2.e. decrease in the uninsured.

3. For #3 Behavioral and Mental Health, relating to decreases in a. teen pregnancy rate, b. unplanned pregnancies, c. substance use, d. domestic violence, e. child and elder abuse, recommending deleting all these measures. Again, we anticipate that vast majority of the network does not have activity/initiatives in these areas. The collection of this information is extensive and difficult to obtain. If there is a desire to collect some information on efforts, then include a narrative that can be reported on with some basic information requested.

4. For #4 Public Safety, a.-d., relating to the decrease in recidivism rate, non-violent crime rate, violent crime rate, juveniles involved in court system, we anticipate that the vast majority of the network does not have activity/initiatives in these areas. Recommend deleting. Relating to # 5, it may be a possibility for some larger communities that participate in development/enhancement of emergency management plans. However, the information requested is extensive and difficult to obtain. If there is a desire to collect some information on these efforts, then consider collecting information in a narrative format.

MODULE 3, SECTION B

Civic Engagement and Community Involvement Goal 2

1. Recommend deleting #1 thru #4 which measure percent increase in donated time, resources, people participating in hearings and forums, etc. There likely is not great variance from year to year. It should be sufficient to report the outcomes in Module 4 for these types of activities and not necessary to also report information in Module 3. It may not be necessary that every Individual and Family Level outcome also have a related Community Level NPI.

Civic Engagement and Community Involvement Goal 3

1. Recommend deleting #1 thru #3 which measure percent increase in low-income people who support community action and increase leadership roles and other. The numbers reported are fairly low and there likely is not great variance from year to year. It should be sufficient to report the outcomes in Module 4 for these types of activities.

Comment 118 (The State Of New York Community Action Association ; State Association in NY)

Module 3: Module 3 – Community Level: (pages 28-50)

· Collective impact is a set of strategies and term of art currently used to describe efforts to engage in community level work but is neither the only nor preferred option and should be removed from the reporting tool.

· Social or population indicators are unnecessary and provide little useful information on the impact of community level initiatives and strategies. As noted above, reporting and evaluating the success of community-level interventions requires a rigorous evaluation that accounts for the influence of external factors and allows for the long time period in which community-level indicators are undertaken. An effective and useful assessment should include the ability to report over the long time frame within which the intervention occurs, the types of strategies employed, the target population and the outputs achieved during the long process on implementing a truly effective community-level initiative.

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

· As noted above, it is recommended that the Collective Impact strategy be removed.

Module 3, Section A: Supplemental Data: (page # 32)

· Remove Collective Impact

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

· Including the percent in this section is highly misleading and may result in unduly negative assessments of the impact of a community level intervention. It is difficult to identify a “community” and find corresponding data, for example for a community education project. Furthermore the percentage may be misleading, for example creating 25 day care slots in a community with a high unmet need for these slots may not look like a successful undertaking when in fact it achieved the realistically goal set. .

Comment 119 (TN Department Human Services ; State in TN)

Module 3: Module 3 – Community Level: (pages 28-50): Overall I think all would agree that it’s great how the Community Level Initiatives have been outlined and would track an agency’s overall progress. In addition, all would agree that it would promote better outcomes for individuals and communities throughout the network. The biggest concern though, in particular for our state, will be the additional burden to thoroughly train eligible entities on how to report their community level outcomes. Again, timing and cost is a huge issue for our state. The only other concern is how to determine what your baseline should be? Is that based upon evidence-based models, your local community needs assessment, or what?

Comment 120 (Tompkins Community Action ; Eligible Entity/Local Agency in NY)

Module 3: 3. The report on community strategies is another example of listing descriptions and inputs or outputs, not outcomes, and should not be required

4. The multi-part community project outcomes reports in their present form should also be excluded. We believe a simple set of reports can tell a story of community level work. The information collection systems proposed are tools for project management on the whole. Further, the sections of extensive reports on the nature of partnerships and the elements that identify collective impact partnerships are not related to the NPIs and organizational standards currently in effect. Finally, we believe that many of the proposed community indicators create a layer of paperwork that is not useful.

Our respondents almost universally indicated that the proposed community indicators involving demographic impacts and economic impacts on communities are inappropriate. It is also impossible to measure some conditions at a small geographic scale because projects are affected by externalities not within the control of any agency. What is important is achieving the goals of the project as determined by the Community Action Agency board and approved by the state.

Comment 121 (Total Action For Progress ; Other/Unknown in VA)

Module 3: Reporting on community-level outcomes, at least as described in the proposed requirements, is neither feasible nor helpful.

Comment 123 (Tri County Community Action ; Other/Unknown in MN)

Module 3: Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

Remove Collective impact as it will not demonstrate performance of a single agency as local agencies implement different programs so data will be unclear as to performance due to other community factors.

Rates and percentages of changes will not show viable outcomes or impact of a local agency and need to be measured by different variables because data could be viewed as unfavorable.

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

Local agencies currently have the ability to determine their own strategies based on their individual community needs. A narrative on justification would be viewed as a negative towards specific communities and agencies.

Comment 124 (U.S Department Of Health And Human Service ; Eligible Entity/Local Agency in DC)

Module 3: Module 3 — Community Level

In this module, all of the new requirements would require additional staff time and effort for the State CSBG Office and our eligible entities to track, collect, and report this data and information. Training and technical assistance for the State CSBG Office and our eligible entities would be necessary and could be costly; and our eligible entities would have to pay for their costs.

Section A: Community Level Initiatives in Progress

This proposed report is new and would work for showing the progress eligible entities make towards their community level initiatives, but it is unlikely it would be useful beyond that, and it is not necessary for eligible entity performance. Our State CSBG Office and eligible entities agree that progress on community level initiatives should be reported, but the amount of information that this proposed report currently requires adds an unnecessary amount of burden on the eligible entities. This proposed report should be reworked and streamlined to lessen the burden on eligible entities.

Section B: Community Level NPIs

In regards to requiring eligible entities to report baseline numbers: This new proposal would require an enormous amount of training and technical assistance for the State CSBG Office and our eligible entities. Our eligible entities are unsure how such specific baseline data would be obtained, understood, and contextualized.

In regards to requiring eligible entities to report targets: Our state sees the Community Level NPIs Report as a tool for reporting and showing eligible entity community level outcomes (telling the story), not as a planning and analysis tool. Our eligible entities stated that they use their client and community needs assessment data, program specific data, and other local data and information for program planning and implementation; rarely do they use the data and information from their Community Level NPls Report for planning and analysis.

The State CSBG Office and our eligible entities agree that eligible entities should report their community initiatives outcomes annually to the state and OCS. Therefore, we suggest the baseline numbers and targeting component be removed from this report and the report focus on collecting community initiatives data and information solely.

Comment 125 (Vermont Community Action Partnership ; Other/Unknown in VT)

3. Module 3: Remove the Community Outcome Indicators that include percentages or rates. Their inclusion is inappropriate and will provide meaningless or even misleading data when aggregated beyond individual initiatives. Such data has validity for geographic areas and populations much larger than those addressed or targeted by local agency services and initiatives. This validity is lost when the scale is smaller and generally accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. In fact, it is more likely to end up being detrimental to our purposes, since it will more often under-report the actual outcomes and impact on the people the agencies are actually serving when they are compared to an inappropriately large community-wide population that most agencies have no realistic capacity to make a measurable impact on. It is much more appropriate and useful to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

4. Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work, and while EEs are engaged in such efforts, privileging one approach over others, given the broader range of options for doing community-level work, is inappropriate and unnecessary. We are suggesting that Collective Impact is not a worthwhile strategy for many agencies to engage in. We’re just recognizing the reality that relatively few of us have sufficient resources and/or are in a position to mount an effective initiative of that kind; consequently, making it an expectation in the core reporting tool will likely create the unfortunate perception that the majority of agencies are failing to meet a key expectation and thus diminish the general perception of the entire network’s effectiveness. This should be removed and instead encouraged as part of training and technical assistance initiatives.

Module 3: Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Community Level Initiatives Home Page: (page # 29)

1. Remove all collective impact options from the reporting tool. The data to be reported is unnecessary for the performance of local agencies. It will have no utility at the state and national levels as described currently, and could produce unintended negative consequence of making it look like our network is ineffective in this area since there are so few agencies reporting on it. Collective Impact is a specific set of strategies for engaging in community-level work and while it should be encouraged in other ways, it is counterproductive and misleading to position this approach over all other options.

2. Remove all rates and percentages. Reporting changes in rates of social or population indicators is inappropriate, unnecessary for proper performance of the state and federal agencies, and will not provide clarity to the impact of community-level initiatives and strategies implemented by EEs and their partners. These elements should be, and are, used in community needs assessments, not as an outcome indicator for agencies to report to state LAs. Community-level indicators are influenced by a wide range of factors and even successful agency initiatives will most likely show little impact, meaning the data is of limited utility and potentially misinterpreted in negative ways. Accurate determination of the outcomes and impact of many types of community-level initiatives requires rigorous evaluation and cannot be assessed by reporting community-level indicators alone.

3. A more appropriate system may allow local agencies to report: 1) the overall domain of the community initiatives (e.g. health, workforce, etc.); 2) the types of strategies employed (e.g. campaign to increase the minimum wage, EITC, etc.); 3) the target population (e.g. all at-risk high school students in a three county area); and 4) Outputs or Counts, which would allow for agencies to get credit for activities that move the initiative forward but hadn’t yet resulted in quantifiable outcomes.

Module 3, Section A: Community Level Initiative Status Page: (page # 30-31)

Remove Collective Impact (See above)

Module 3, Section A: Supplemental Data: (page # 32)

Remove Collective Impact (See above)

Module 3, Section B: Community Level NPI Landing Page: (page # 37-39)

1. Remove “and percent” from all community level indicators, for the reasons indicated above. Even though agencies are allowed to define what constitutes “community”, it may be difficult for them to collect this data if it doesn’t correspond to a unit or level of analysis for which there is readily available data (e.g., if community is defined as several census tracts for which the specific data in question is not available). Also, because the definition of what constitutes the community is left up to the agency, they may struggle to define it exactly (e.g., in the case of a broad public awareness campaign) which is unduly burdensome and may result in the collection of data with limited utility. Finally, it may unfairly force a comparison between the agency’s outcome and the overall rate of a particular community level indicator. For example, a county-wide workforce development initiative may successfully create 100 jobs, but may look (misleadingly) like a minimal result when compared with the overall county unemployment rate which is affected by many other forces.

2. Rates and social indicators are not appropriate for a reporting instrument from a local agency to the state office. These elements are better used as part of a needs assessment. Remove all indicators that depend on measuring and impacting “Rates” as a core element, specifically:

o Education and Cognitive Development: 5, 6, 7, 10

o Housing: 6, 7, 8

o Health and Social/Behavioral (2a, 2b, 2c, 2d, 2e, 3a, 3c,)

o Public Safety (4a, 4c)

Module 3, Section B: Community Level NPI Data Entry Forms: (page # 40-50)

This instruction is misleading and is inappropriate. It will negatively impact the capacity of agencies to perform their functions properly. Local CAAs have the ability to determine the strategies they implement in the communities they serve and to have the reporting tool state, “agencies must provide a narrative justifying….” is inappropriate.

Comment 126 (Vernon Community Action Council, Inc.; Eligible Entity/Local Agency in LA)

Module 3: No Comment

Comment 127 (Virginia Community Action Partnership ; State Association in VA)

Module 3: Regarding Module 3, we reiterate the comments that we made about community outcome indicators in our preliminary comments. We are deeply concerned that the collection of data by percentage or rate will not yield useful data and does not appropriately reflect the scale of local agency projects. It would be more appropriate to give agencies the option of reporting outcomes data and the flexibility to define which indicators they use. Data produced by the agency would better reflect whether the agency activity contributed to a specific outcome.

Comment 129 (Western New York Community Action Agency; Eligible Entity/Local Agency in NY)

Module 3: No Comment

Comment 131 (Wisconsin Department Of Children And Families; Eligible Entity/Local Agency in WI)

Section A: Strategies Popup Windows: (page # 33-36)

p. 34 – Housing Strategies – I’d recommend including permanent affordable housing units to the second item on the list, which currently reads, “Create New Affordable Housing (SROs, temporary housing, and transitional housing.

Comment 132 (Work And Family Support Bureau; Other/Unknown in NM)

Module 3: No Comment

Comment 133 (WSOS Community Action; Eligible Entity/Local Agency in OH)

Module 3: No Comment

Comment 134 (York County Community Action ; Eligible Entity/Local Agency in ME)

Module 3: * Remove collective impact from all reports. Collective Impact models are certainly used by our organization, however, we also believe that partnerships can manifest and provide benefits to clients in many different forms.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Anthony Davis
File Modified	0000-00-00
File Created	2021-01-23