General Comment Burden

General Comment - Burden

Comment 1 ( Arizona Department Of Economic Security, Division Of Aging And Adult Services, Community Action Programs And Services (Daas/Caps) ; State in AZ)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The State CSBG Office, DAAS/CAPS, shares the concerns of the Community Action Network in Arizona, that the Module 4, Sections A and B (Characteristics for New Individuals/All Characteristics Reports) instructions are unclear regarding when data is to be collected on each. There is also concern regarding the usefulness of this data, considering most clients applying for services will be new in the current reporting period, and many will likely not return in the same reporting period (unless they are participating on ongoing programs). The number of returning individuals in the same reporting period is probably statistically insignificant. These reports will tell us very little about what has changed regarding these individuals over a single reporting period covered by both reports. It is the opinion of the State CSBG office, that these comparisons would only be useful if applied over time. This could be accomplished by comparing the All Characteristics report from year to year.

With regard to the community level data, Module 3, Section B: there needs to be clarification regarding the numerous questions that ask “percent of” specific outcomes. These questions do not clearly identify the variables by which a percentage is determined.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

The estimated burden of collection may be too low, considering there is follow up time involved in data cleaning and data validation, and technical assistance needed.

3. The quality, utility, and clarity of the information to be collected; and

Module 4, Sections A and B may not provide the intended information due to insufficient duration of observation. CAAs often see clients only once in a program year, therefore two demographic reports (Module 4, A & B) are not likely to provide useful information. See Module 4 for more detail.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Assistance to States to provide the CAAs with mechanisms to modify their data collection systems and collect the new information would be helpful

Comment 2 ( Clarity, Impact And Performance Project Steering Committee; RPIC in ID)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

* Cost effectiveness of the new system: The information obtained from this report that will be meaningful and useful at a local and/or national level will not justify the effort required to complete the data collection and submission for the Annual Report. From Region 10’s response from the first comment period: “Despite the fact that RNG has essentially taken us from 6 national goals to 2 (with the civic engagement goal from the infographic apparently being included under both the individual/family goals and the community goal and not a separate third goal), the data collection and reporting have become more complex. If the 6 NPIs and the past reporting process wasn’t getting us data that was reliable, how does increasing the complexity help? More data doesn’t equate to more effectiveness, especially given the questions around the capacity to collect and report data with integrity.”

* Since the close of the first response period, changes were made to some of the indicators, however, the complexity remains high. With complexity comes room for a variety of interpretations. Will CAAs in each state, much less around the country be able to agree on the meaning of each indicator? A shared understanding and meaning of the data is required in order to have any value.

* We understand that social indicators have been removed, for now, from RNG. These indicators are used to describe the conditions in local communities and should be left to the federal, state and local jurisdictions currently responsible for gathering and maintaining this data and information.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

* OCS provided the network with an estimate of how long it will take a CAA to complete this new report. It would be helpful to know the methodology used to arrive at the estimate to determine if there is a basis in the reality that mirrors the very busy, day to day operations of a community action agency.

* Beyond this estimated time to complete the survey, has any effort been made to analyze the capacity of the network to report data with the level of complexity outlined in the proposed reporting format with any degree of validity?

* Staff responsible for reporting using the current IS Survey process estimate that the new format will at least double and probably triple the time it takes to do a good job with this proposed process. That increased time comes with no additional offsetting resources, and will take time and resources away from providing mission driven services in communities.

* We do not believe that a majority of the CAAs in our 3 states have the capacity (staff, financial, technical) to report with this level of complexity. We are particularly concerned about rural agencies. For some regions in our states, the CAA is the primary providers of services to low-income individuals. The burden of these reporting requirements will significantly reduce the time and resources to provide services.

* A majority of the CAAs in our 3 states do not have the capacity to report unduplicated numbers of people receiving services by domain. Head Start, energy and weatherization, housing and homeless services have separate data reporting systems in Oregon, for example. Asset building has yet another. Service Point is used by Continuum of Care and VA programs. This issue is likely not unique to the 53 agencies represented by this response.

3. The quality, utility, and clarity of the information to be collected; and
* The cart is out in front of the horse. For the data to be valuable at the national level, and to warrant the time and resources required, there must be an understanding of clear and shared definitions. There has not been enough time devoted to developing this understanding at the local or national level. It would be impractical and unwise to implement RNG and then attempt to get the national network in agreement about what the data elements mean and how data should be collected.

* A common understanding of each Community Measure will be required if the data from these measures will provide any valuable information. How will we as a statewide or regional network of CAAs come to a common understanding? Will Region 10’s understanding be the same as other Regions? If not, how will the data gathered be useful (see bullet #1)?

* Without a shared understanding, with agreed upon definitions, CAAs will spend days and weeks collecting data from multiple sources in order to fill in the blanks and check the boxes. As IT/data staff will tell you, “garbage in, garbage out.”

As stated in the response to #2, the quality of data will be undermined by the sheer volume of systems that the data must come from. If the data quality is mediocre at best, then the utility and clarity is compromised.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

* It is possible to collect information from clients of CAAs by using iPads or laptop computers, located at a reception or work desk. This works best for clients who have received a very specific service (e.g. energy assistance, rental assistance, budgeting classes) and are able to relate the questions to that specific service. At a minimum this requires:

1. That staff understand the importance of the survey and gently encourage clients to take it;

2. That clients are willing to take it;

3. That staff are available to assist clients who may have a language or reading barrier, without contaminating the survey process;

4. That staff are available to monitor the devices;

5. That staff are available to deal with problems that may arise with the iPad or laptop;

6. That funds are available to purchase and maintain the equipment.

* OCS provided the network with an estimate of how long it will take a CAA to complete this new report. It would be helpful to know the methodology used to arrive at the estimate to determine if there is a basis in the reality that mirrors the very busy, day to day operations of a community action agency.

Our estimates for agencies who have under 100 employees and those who have over 100 employees are as follows:

Hours per week: Over 100 Staff Hours per week: Less than 100

Year 1 Year 2 Ongoing Year 1 Year 2 Ongoing

51 59.5 48.5 40 43.5 36

Hours per year: Over 100 Staff Hours per year: Less than 100

Year 1 Year 2 Ongoing Year 1 Year 2 Ongoing
2652 3 094 2522 2080 2262 1862

* A majority of the CAAs in our 3 states do not have the capacity to report unduplicated numbers of people receiving services by domain.

* The OMB could require that the VA, HUD, OCS and the Department of Education allow crosswalks to be developed so that OMB can look at outcomes across programs without placing this burden on Community Action.

* Rather than spend time and resources on rolling out Roma Next Gen, the OMB could require that all government funders of social service programs collect data on the same indicators (or at least some of the same indicators).

* To test the viability of the RNG indicators, OMB might require OCS to start out with a small sample, with a focus on one area of service (e.g. housing stability).

Comment 3 ( Hill Country Community Action Association, Inc. ; Eligible Entity/Local Agency in TX)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

Most of the information collected is not necessary for the proper performance of the agency and will not be utilized in any manner by the agency.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

It is not possible to accurately estimate the burden of the proposed collection of information, but it is very disheartening to think of the time and resources spent on data collection and other unfunded mandates. We are already spending more dollars directly and through staffing to meet all the current regulations and adding another layer of reporting requirements is not in the best interest of the clients we serve. Agencies are already strapped with so many mandates that it has put a burden on staff at all levels to keep up day to day with task unrelated to assisting our clients. For the amount of money most Agencies receive it is not possible to provide anywhere near all the reportable services or have a reportable impact on the communities we serve because in most cases we are the smallest player at the table.

3. The quality, utility, and clarity of the information to be collected; and

Each Community Action Agency is somewhat unique in that by law it is locally controlled. This creates a multitude of services that could be provided. It is impossible to create a data base that can capture all the achievements and projects of all the agencies. Most likely the information collected will not show the true picture of what a Community Action Agency is doing and will not tell the true CSBG story.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Those of us who are fortunate enough to be able to afford a data base program to collect data for the current annual report will have to come up with additional funds to upgrade the systems. Those who at this time do not have an adequate data base program will have to spend even more funds to capture the required data. Again this is money that could be better spent assisting clients.

Comment 4 ( Sacramento Employment And Training Agency (Seta) ; Other/Unknown in CA)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

Most of the information collection proposed in the June 2016 Community Services Block Grant Annual Report, is already collected from 19 CSBG delegate agencies, internal CSBG program staff, multiple competitive grant and foundation-funded programs, SETA and delegate Head Start sites serving over 6,000 children daily, refugee and youth services throughout Sacramento County, National Emergency Grant activities, 13 Workforce Innovation and Opportunity Act (WIOA) America’s Job Centers serving nearly 40,000 individuals annually, and the leveraged resources of regional, public and local entities in the Greater Sacramento Area. In each of the programs noted above, data collection is designed to efficiently collect and analyze data elements requested by each independent funder and to provide reasonable oversight and accountability, ensuring that the primary goal of providing community services or resources is achieved.

The desire to overlay a single standardized system of data collection, compilation, analysis, reporting, and identifying unduplicated households accessing multiple services across disparate programs administered by individual CAAs, to the Office of Community Services (OCS), is understandable. What is less clear is the need to add to existing data collection processes and systems that already meet the data reporting requirements of the various federal, state and other programs.

CSBG resources represent less than 2% of SETA’s annual budget. The proposed data reporting requirements would significantly increase the cost of data reporting to the CSBG program and reduce the levels of service provided to the community. Furthermore, the proposed data reporting requirements would discourage community-based organizations and other service providers from participating in the CSBG program thereby reducing the program’s ability to leverage other resources.

The Annual Burden Estimates noted in the OCS Dear Colleague Letter dated June 17, 2016, might seem adequate for a CAA mature and fully-developed reporting process across few systems, but it does not represent the burden of proposed collection of information for SETA. The estimates do not include the increased time and effort in coordinating and reconciling data collection across multiple information systems, ongoing training and technical assistance for over 50 service locations, and the increased administrative costs of information technology, software, equipment and maintenance necessary to manage such a system. SETA is unable to provide any quantifiable estimate of the Annual Burden of the proposed reporting system, but it is clear that the annual burden would greatly exceed the suggested 242 hours.

Suggested options include: (1) using an intelligent software solution that compiles data from multiple data reporting systems, and (2) a centralized federal approach that deconstructs NPI aligned data from national governmental reporting systems and delivers it to the CAAs.

Comment 5 (Action For Better Community ; Eligible Entity/Local Agency in NY)

The practical utility and necessity of the collection of the data requested is extremely limited. The agency currently completes both quarterly and annual data reports for the state oversight agency (NYS Department of State, Office of Community Services). The program outcomes in those reports as they relate to the agency’s community action plan are very useful, indeed, necessary for agency quality assurance and needs assessment. The data that are used for the national CSBG-IS report, however, are not used by the agency for any reason other than required completion of the report itself.

The agency has been collecting and reporting data agency wide as well as at the program level for decades; it is fairly straightforward to estimate the burden of the proposed information collection with some accuracy.

First, the agency does not foresee any reduction in data collection and reporting as described in the proposal. The online interface to collect data by the states, which may be made available to grantees, does NOT promise to offer any time-reduction or lesser burden to report information. The agency currently uses such an interface to enter its annual performance data with the state. The proposed interface would either replace that tool or be used in addition to it. Due to the volume of data required by the proposal, either a replacement or addition of this new reporting tool would result in more of a reporting burden.

More concretely, the agency estimates very conservatively that it utilizes well over $75,000 in annual resources to support the collection and reporting of agency (non-fiscal) data. This figure includes a full-time staff person who coordinates agency wide data reporting, staff time collecting and reporting data internally and annual costs of several different agency databases.

The agency will respond to the issue of resource burden by estimating the ADDITIONAL costs it anticipates as a result of the proposed changes to the ROMA report:

* Staff time to collect intake data would increase slightly due to the requirement to define and track “new” customers. Estimated staff affected = 15 staff. Estimated hours of additional time per year = 4 hrs. Estimated cost to the agency (@ $20/hr.) = 15 x 4 x 20 = $1,200.

* Staff time to transfer data from existing agency tools into the new database/reporting system will increase due to the significant increase in the number of data points being collected. Estimated staff affected = 15 staff. Estimated hours of additional time per year = 12 hrs. Estimated cost to the agency (@ $20/hr.) = 15 x 12 x 20 = $3,600.

* Staff time to develop new training materials on the indicators and to provide training (this would likely be a “one-time” expense as such training is already part of the agency’s reporting budget). Estimated staff affected = 15 staff. Estimated hours of additional time per year = 8 hrs. Estimated cost to the agency (@ $20/hr.) = 15 x 8 x 20 = $2,400.


* Staff time to perform quality assurance of data would increase due to the increased number of measures as well as the increased complexity of the community indicators. Estimated staff affected = 2 staff. Estimated hours of additional time per year = 10 hrs. Estimated cost to the agency (@ $40/hr.) = 15 x 8 x 40 = $4,800.

* Staff time to track data will increase due to increased categories and complexity. Estimated staff affected = 15 staff. Estimated hours of additional time per year = 4 hrs. Estimated cost to the agency (@ $20/hr.) = 15 x 4 x 20 = $1,200.

* Staff time to establish community baselines for multiple community impact initiatives required in Module 3. Estimated staff affected = 4 staff. Estimated hours of additional time per year = 15 hrs. Estimated cost to the agency (@ $40/hr.) = 4 x 15 x 40 = $2,400.

* Staff time to update internal tools for collecting and aggregating data will be required on a one-time basis to transition to the new requirements. Estimated staff affected = 3 staff. Estimated hours of additional time per year = 20 hrs. Estimated cost to the agency (@ $40/hr.) = 3 x 20 x 40 = $2,400.

Note that the above expenses do not foresee updated databases, software purchases or other expenses that might be required to support data collection. This estimate conservatively assumes existing infrastructure can suffice.

The sum of the above expenses is $18,000. The agency considers this to be a very conservative estimate of NEW expenses resulting from the proposal. The agency would not choose, voluntarily, to spend this amount of money to implement this system – it simply does not offer significant return to the agency given the fiscal resources required to support it.

3. The quality, utility, and clarity of the information to be collected; and

The quality and utility of the data has been addressed previously, but to reiterate some key points:

* Number of new customers is not useful information. Instead, changes in the patterns of services received or in the outcomes experienced are more actionable for the agency.

* Changes in # of new customers can potentially reflect contradictory trends with no indication which is in effect: an increase in the number of new customers could reflect harsher local conditions that require more people to seek services, OR it could reflect more effective outreach and collaboration by service providers to connect people with needed services.

* Community level indicators are wrought with complexity. They are often not available for the population of interest. More importantly, the complexity of community level issues means single rates or percentages are capturing the impact of multiple forces. Relying on one data point when one knows that several variable drive that number is not sound practice.

Comment 6 (Action For Boston Community Development Inc.; Other/Unknown in MA)

Many of the changes in the Annual Report requirements will call for modifications in the orientation and day-to-day workflow of agencies, in the interest of greater rigor in data collection. To the extent that these changes are manageable, they are welcome; however, many will create increased time and cost burdens.

* Estimated time and cost. The time and cost burden of reporting appears to be substantially underestimated by OCS at 242 hours per sub-grantee (individual Community Action Agencies). For example, Action for Boston Community Development, Inc. (ABCD) estimates an annual time burden of 771 hours, excluding the data gathering activities required of front-line staff. When the work of these front-line staff members is included, there is a 116% increase in the overall cost of maintaining the systems required to support production of the Annual Report (see attached Table 1)

The overall scope of data to be reported has been dramatically enlarged in the proposed Annual Report, which is the major driver of these time and cost increases for ABCD. However, for many Community Action Agencies the escalation in time and cost burdens will be much higher.

* Investment in data warehousing. ABCD is in the relatively unusual position, among its agency peers, of having invested in data warehousing systems which support data collection across multiple program-specific databases. This is a necessity for meeting the Annual Report requirements. For an organization managing multiple programs to establish such a data warehousing system is a major capital expenditure—and it is a larger expense for the organizations which have built the strongest arrays of programming. For example, ABCD has expended several hundred thousand dollars in creating and upgrading its internal data warehousing capacity. This was an especially complex process, given ABCD’s size and the evolving nature of the systems; however, even a modestly sized data warehousing project represents a major cost (see attached Table 2)

It is important to note that such data warehousing systems provide important benefits for individual sub grantees, and may be a practical necessity for reporting unless and until major federally-funded human services programs utilize interoperable automated client tracking and reporting tools (or at the least, share a common data dictionary). However, they require substantial new resources to establish and manage, and no new funding has been identified.

* OLDC. The projected Online Data Collection system (OLDC) is unlikely to reduce these time and cost burdens, and in fact it is much more probable that OLDC will require additional investment from individual agencies.

Question 3: The quality, utility and clarity of the information to be collected

There is a crucially important distinction to be made among the various information requirements in the proposed Annual Report.

* Improvement in family and individual outcomes. The individual and family outcomes articulated in Module 4 represent, for the most part, dramatic improvement in the quality and meaningfulness of data that can be developed from local agency reporting (there are some specific issues with the items included, which are detailed below.)

However, there are important methodological problems associated with reporting requirements in other areas.

* Issues with “new” clients. As noted above, the inclusion of reporting for “Characteristics of New Individuals and Households” in Module 4, Section A: Data Entry Form is unlikely to be of utility to local grantees, and will produce meaningless numbers, at best, if aggregated on a national level.

* Issues with community measures. Similarly, the way in which “community-level” data is used in Module 3, Section A is very likely to produce reported numbers which are of limited use in program management, and inappropriate for national aggregation.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

The most important immediate action that HHS can take to reduce the burden of data collection is to eliminate the elements of the proposed Annual Report which will generate data that has little or no utility.

However, this review process highlights the need for HHS-funded programs to develop solutions to issues of data integration—whether by creating interoperable program reporting systems, or by providing assistance to individual local agencies in the development of data warehousing systems.

Table 1: TIME AND COST ESTIMATE FOR ANNUAL REPORTING

ITEM CURRENT Hours/ Cost PROPOSED Hours/ Cost OVERALL CHANGE

1.CLIENT DATA MANAGEMENT

a. Case Worker Level (Gathering and entering data on households)

b. Planning Staff Level (Cleaning ,data, coding)

c. Data Warehouse (Adding and updating merge algorithms, other maintenance) a. Staff currently report 2 hrs./wk./person @ $20/hr. for 40 persons = $83,200 b. Staff currently report 22 hrs./mo. @ $28/hr. = $7,392 c. Current contract cost averages $10,000/yr. TOTAL COST $100,592 a. Test suggests increase to 4.5 hrs./wk./person @ $20/hr. for 40 persons = $187,200 b. Test suggests increase to 26 hrs./mo. @ $28/hr. = $8,736 c. Contract cost estimated at $16,000/yr. TOTAL COST $211,936 $ 111,344

2. STAFF TRAINING

a. Case Worker Level
b. Planning Staff Level
c. Data Warehouse a. Staff currently receive 2.5 hrs. Annually @ $20/hr. for 40 persons = $2,000 b. Staff currently average 4 hours annually @ $28/hr. for 3 persons = $336 c. No current charges TOTAL COST $2,336 a. Projected at 3.5 hrs. annually @ $20/hr. for 40 persons = $2,800 b. Projected at 8 hours annually @ $28/hr. for 3 persons = $672 c. Additional costs of approximately $4,000/year projected, associated with community program tracking and reporting TOTAL COST $7,472 $5,136

3. DEVELOPMENT AND TRACKING OF COMMUNITY MEASURES
a. Identification of community a. No current charges a. Projected at 12 hours baselines b. Tracking of community projects activity and impact b. No current charges TOTAL COST $0 annually @ $28/hr. for 3 persons =$1,008 b. Projected at 16 hours annually @ $28/hr. for 3 persons =$1,344 TOTAL COST $2,352 $2,352

4. PREPARATION AND SUBMISSION

OF ANNUAL REPORT
a. Communication and coaching with internal departments and
delegate agencies

b. Collation of reported data


c. Drafting required narrative information

d. Drafting required Organizational Standards information

e. Data quality assurance

f. Entry into state portal a. Staff currently estimate 5.5 hrs. annually @ $28/hr. for each of 12 reporting units =$1,848 b. Staff currently estimate 6 hours annually @ $28/hr. =$168 c. Staff currently estimate 6 hours annually @ $28/hr. =$168 d. No current charges e. Staff currently estimate 6 hours annually @ $28/hr. =$164 f. e. Staff currently estimate 6 hours annually @ $28/hr. = $168 TOTAL COST $2,520 a. Projected at 12.5 hrs. annually @ $28/hr. for each of 12 reporting units =$4,200 b. Projected at 14 hours annually @ $28/hr. =$392 c. Projected at 10 hours annually @ $28/hr. =$252 d. Projected at 18 hours annually @ $28/hr. =$504 e. Projected at 22 hours annually @ $28/hr. =$616 f. e. Projected at 10 hours annually @ $28/hr. =$280 TOTAL COST $6244 $3,724

TOTAL INCREASE $122,556

Table 2: TIME AND COST ESTIMATE FOR DATA WAREHOUSE CAPACITY DEVELOPMENT

TASK INPUTS COST

Non-Recurring Costs

1. INITIAL REVIEW AND COLLATION OF DATABASE ELEMENTS AND

REPORTING REQUIREMENTS a. Internal MIS Staff 10 hrs. @ $40/hr. = $400. b. Internal Planning Staff 4 hrs.@$28/hr. = $112. $512.

2. CONTRACTING AND SPECIFICATION FOR DATA

WAREHOUSE PRODUCT AND SERVICES a. Internal MIS Staff 18 hrs. @ $40/hr. = $720. B. Internal Legal Staff 2 hrs. @ $45/hr. = $90. $810.

3. PERMISSIONING ACCESS FROM

DATABASE MANAGERS a. Internal MIS Staff 18 hrs. @ $40/hr. = $720. $720.

3. LICENSING OF DATA WAREHOUSE PRODUCT a. Internal MIS Staff 6 hrs. @ $40/hr. = $240 b. Licensing Fee/Purchase @ $50,000 $50,240.

4. MERGING DATA SETS

a. Consulting Services by Data Warehouse Vendor @ $50,000. $50,000.

5. STAFF TRAINING

* SQL Training for MIS Staff

* Training for Planning End-

Users

a. Internal MIS Staff 6 hrs. @ $40/hr. =$240 b. SQL Training @ $2,500 $2,740.

TOTAL NON-RECURRING COST $105,022.
Recurring Costs

1. DATA STORAGE (web-based) a. Web-based Storage @ $2,000/mo. For 12 mos. = $24,000 $24,000

2.DATA WAREHOUSE REVISION AND UPDATE a. Consulting Services by Data Warehouse Vendor @ $10,000. $10,000

3.REPORT WRITING a. Internal MIS Staff 6 hrs./mo. @ $40/hr. for 12 mos. = $2,880 $2,880

TOTAL RECURRING COST $36,880

Comment 7 (Allegany County Community Opportunities And Rural Development, Inc. (Accord); Eligible Entity/Local Agency in NY)

Specific commentary is provided below and in some cases is taken directly from comments prepared by the Community Action Partnership based on our full agreement with much of the feedback they have provided. This also incorporates review of the response provided by David Bradley, Chief Executive Officer of the National Community Action Foundation. While we support the overall content of his response, we are especially agree with his reference to the tremendous burden the collection of this information will impose. As Mr. Bradley noted, our funders, public and private, require information to be provided via their own customized requirements and identifiers. We find increasingly, that funders are demanding more segregation of data, outcomes and expenditures to assure that their investments are in no way benefitting another program or service. Our goal is to measure our true impact on our families and communities. We believe ROMA NG can and should move forward to support that goal but believe the requirements of this report places the burden on State CSBG Lead Agencies and CSBG eligible entities alone. We have neither the resources nor leverage to make this collection of information truly possible and believe the results may lack useful application and clarity.

Comment 9 (Association Of Nationally Certified Roma Trainers ; Eligible Entity/Local Agency in N/A)

Question me. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

Yes — this report "closes the loop" related to several OCS initiatives: the Model State Plan,

State and Federal Accountability Measures and Organizational Standards and modernizes the Information Survey currently in use for reporting on agency capacity, use of resources, demographics of individuals and families served, and achievement of outcomes. As such the practical utility is to have a vehicle for reporting on key elements of an extensive performance management system for all levels of the CSBG network.

While we hear that the reporting of services and strategies will create a burden for some agencies, we applaud the separation of services from outcomes, which has caused confusion in the past. It is important for agencies to be able to identify what they do as well as what happens as a result of their work. This report will allow that and will have practical application.

Comment 11 (California Community Action Partnership Association; Other/Unknown in CA)

On behalf of the 60 member agencies that comprise the California Community Action Partnership Association, I am writing to respectfully submit comments on behalf of our collective membership regarding the proposed Community Services Block Grant (CSBG) reporting requirements familiarly known as “ROMA Next Generation”.

Your office has received many responses from our individual member agencies so I will not be voluminous in presenting the concerns of our member agencies, however, it is important that the main issues are reiterated in order for our Federal partners to understand the tremendous burden this will potentially place on Community Action Agencies at the local level. The issues I will note are reflective of the feedback CALCAPA has gathered from our network agencies.

Additionally, I write from the perspective of a professional who has provided leadership at the Municipal, County, and State levels in implementing performance-measurement and management tools and systems. I helped launch Baltimore City Stat to measure and improve performance of City Departments and service delivery. As the Assistant Chief of Juvenile Probation for Dallas County Texas I created JD Stat as a performance management tool for the largest Juvenile Justice agency in Texas. In my role as Executive Director of the Community Services Administration for the State of Maryland, I served as a leader on the team assembled to implement the Maryland State Stat Program which received national acclaim.

All of our agency leaders agree wholeheartedly that we must utilize data to better measure effectiveness as well as to tell Community Action’s story more poignantly and effectively to policy makers and the public at large, however, our members have expressed serious concerns about ROMA Next Generation being the tool to accomplish that goal! Some of the major issues you have already been made aware of that give our member agencies pause in moving forward with ROMA Next Generation and the proposed CSBG reporting requirements are:

1. COST AND FINACIAL BURDEN ON LOCAL AGENCIES

One of our larger more complex agencies after conducting an extremely thorough analysis of what it would cost to purchase and maintain a system to collect, collate, and report unduplicated data across our 14 programs, 49 funding sources, 101 contracts, and 42 funding periods was estimated in excess of $270,000 per year.

2. THE ADMINISTRATIVE TIME BURDEN


As well as the intentions may be with the proposed reporting requirements, it does not factor in the burdensome time to collect the desired data. One of our smaller Limited Purpose Agencies recently concluded after a deep diving analysis it would require
12,509 hours for staff to compile the required data.

3. DOES NOT FACTOR THE UNIQUENESS OF EACH COMMUNITY ACTION AGENCY

It is often stated that if you mirror any two Community Action Agencies side by side, you will NEVER find two Community Action Agencies that are identical! It seems clear that the underlying agenda behind the introduction of ROMA Next Generation is in essence contrary to the authorizing language of the Economic Opportunity Act of 1964 and the successive re-authorizations of that Act. The framers of the Act understood inherently that each community is different and therefore they have different issues which require innovative and customized approaches to address. Now is the time to innovate, now is the time to open the immense pools of thought and ideas to demonstrate the value and worth of Community Action. Now is the time to herald our core values – the uniqueness of each community and the leveraging of CSBG dollars to innovate new localized solutions with greater impact and superior outcomes. Each local jurisdiction should determine and prioritize its own needs, then create informed, effective, and tailored responses addressing those needs. Now more than ever is not the time to turn back from the framers intent.

4. ADDED BUREAUCRACY AND OVER REGULATION

Often we hear this point echoed in the private sector but this also is an ever increasing reality for Community Action Agencies, particularly if this reporting framework moves forward. As government adds to the layers of administration and bureaucracy entailed in Community Action, we face a perilous slippery slope towards stripping agencies of the local control that is at the core of our success. Each community, each region in our nation is unique. It is the hallmark of Community Action to empower local leaders and consult local residents to design and develop customized programs and initiatives to fight poverty.

At a time of reduced funding for poverty efforts at the Federal level and harsher conditions facing people with low incomes, it is unconscionable to prioritize data collection and the increased time and expense that will result from the implementation of ROMA Next Generation. The current system is more than adequate. The United States needs a Community Action Network that addresses local needs with a laser focus on innovation and the leveraging of all possible resources to create and introduce opportunities that lead to independence and prosperity among our disadvantaged populations.

For 50 years we as a nation have waged a War on Poverty. As an Association we share the insight of Napoleon Bonaparte, French Military and Political Leader when he stated, “War is ninety percent information”. It is clear that we must more effectively utilize data in our War on Poverty. However, we also share the thoughts of famed Physicist Albert Einstein when he said “Not everything that can be counted counts, and not everything that counts can be counted”.

Thank you for the opportunity to weigh in on behalf the 60 California Community Action Agencies who are fighting every day to empower those who count to lift themselves out of the deep recesses of poverty to enjoy the promise and opportunity that is America!

Comment 12 (Cap Service ; Other/Unknown in WI)

I am the President and CEO of CAP Services, now recognizing its 50th year of service like so many in the community action network. We are proud of the long-term powerful impact generated in the communities we serve, currently through nearly 30 different programs to advance our mission "to transform people and communities to advance social and economic justice." In 2015, we served approximately 2,500 households in our five-county footprint in Central and East Central Wisconsin and have done it transparently and with accountability to our constituents, donors, investors, community partners and volunteers.

A diverse organization like ours is critical to addressing the complex issues of poverty. And while we have been very successful in securing the resources to create and sustain innovative programs, the requirements of our funders are also expansive. Many require significant reporting, often funder-specific reporting tools. Examples of those with such include Head Start and Early Head Start, the Low-Income Weatherization Assistance Program, AmeriCorps, DOL-Youth build, WIOA, SBA, HUD and even more.

We recognize this as a part of doing business but do also ask that you recognize the implications of more expansive reporting for CSBG, including areas not previously required. CSBG is important to our organization but at the same time it is a funding source, not a program. As I posed in my previous comments, this diversity of sources requires significant reporting, often times in slightly different ways.

We have looked at very expensive data systems, and have yet to find one that would adequately, efficiently and cost-effectively move to a central system for tracking outcomes. As such we collect and aggregate information for CSBG reporting, duplicating effort almost without exception because other funders will not waive the requirements to report in their systems (many web-based and not compatible with other systems).

We agree with the Partnership and others in our network that while there are elements of ROMA Next Gen that may be helpful in strengthening our ability to communicate our impact, the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). I question who you identify as the target audience for the report, and whether what is collected both clear and concise enough not to raise questions and doubt the capacity or deliverables by the network (i.e., questions about collective impact or community indicators). I daresay, the breadth of the proposed report would not be particularly helpful to the Congressional delegation nor for the public at large, particularly when you get past individual or household indicators.

We have engaged in local community impact initiatives and see mixed results in terms of true collaboration, relevant and meaningful sharing of data and continuity for the purpose of longer-term measurement. These groups take time to storm, form, norm — and with changes in leadership and time, can fade, or at least lose some of the meaning or spirit for which they were created. To create a system of reporting of the same, without better network planning, analysis and benefit seems short-sighted and ill-advised investment of precious resources.

However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

Comment 13 (Cattaraugus Community Action, Inc.; Eligible Entity/Local Agency in NY)

1: Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

We have carefully reviewed the talking points issued by the National Community Action Partnership and submitted to OCS. We strongly agree with those talking points with respect to the necessity and utility of some of the data elements. We also endorse their recommendations for modification of Organizational Standards, removal of the reporting on characteristics for NEW Individuals and Households, Removal of percentage/rate indicates for Community Outcomes; removal of Collective Impact reporting, the removal of stability indicators, and re-introduction ROMA goals centered on agency capacity.

2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.

We have reviewed available information on time and cost burden associated with implementation of ROMA Next Gen. While we believe that the estimates provided thus far are essential to understanding the scope of the problem, but we further believe that it is not possible to accurately assess the increased time and cost burdens associated with proposed changes. It is apparent that many agencies lack current capacity to report on proposed ROMA Next Gen data points. Correcting this problem will require new/updated reporting software. It is our understanding that HHS is developing software needed to submit state reports, but that this software may or may not be available to local agencies. Without this knowledge, it is not possible to accurately estimate the costs of software design/redesign.

It is also apparent that the new ROMA Next Gen data points will place an increased burden on agency resources with respect to staff time spent in gathering and entering data, monitoring data quality, and generating and submitting reports. Again, it is not possible to place a dollar amount on staff time without a full picture of the data collection system. We can anticipate substantially increased costs associated with staff training in ROMA Next Gen, as well as in data collection, data entry and reporting, but placing a dollar amount on these costs is pre-mature.

3: The quality, utility, and clarity of the information to be collected

We strongly agree with the National Community Action Partnership with respect to their comments on the quality, clarify and utility of the data to be collected and refer you to that document for detailed information. We wish to emphasize that tying a community indicator directly to the work of a Community Action Agency grossly ignores the fact that multiple factors can impact any one community condition. It also sets Community Action agencies up to fail because, despite our best efforts, community factors beyond our control can negatively impact achievement of community based outcomes.

4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Again, we agree with the National Community Action Partnerships suggestions. We further suggest that OCS confer with the vendors of the various report systems currently in the Community Action network, and gather their thoughts on cost estimates and ways that data collection and reporting can be streamlined

Comment 16 (Central Missouri Community Action ; Other/Unknown in MO)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The overall concept of the information to be gathered for the purpose of validating the work of community action is absolutely necessary. Central Missouri Community Action is prepared to meet new requirements of information collection and appreciates the work that has been done thus far. Although we as a state feel comfortable with the idea of ROMA Next Generation, we are concerned with the overall impact to the network if other agencies and states do not have the resources or training to collect information within the proposed framework of indicators and requirements.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information


The time burden seems underestimated. The level of expertise to conduct some of the proposed data collection and analysis is heightened as well. Additional skill sets may be needed, which would increase the overall cost burden associated with the CSBG Annual Report.

3. The quality, utility, and clarity of the information to be collected; and

Quality: The proposed set of indicators seems to focus more on quantity rather than quality. Our recommendation would be that the number of general indicators are decreased (although we know not all agencies must report on each one). If a smaller number of indicators were used to aggregate data to the state and national level, this may be more meaningful to decision makers. Agencies could then use local indicators to track additional outcomes based on their unique programs. For example, if there were 2-3 indicators relating to each of the domains of CSBG these could provide meaningful aggregated data. The local analysis expectation could then be strengthened to fuel the ROMA cycle within the evaluation and assessment portion.

Utility: To the point above, we feel the information would be most useful in fewer, more meaningful indicators that can show national impact of community action in a more concise way. Also, it would seem appropriate to use the language of other indicators that are already established through Head Start or HUD to promote further consistency. A request from our network is to be able to access the local, state, and national comparison data after compiled to use and share locally. Clarity: We respect the strengthened focus on community and believe it is necessary. However, many of the proposed community indicators could be confused with individual or family level indicators. The meaning tends to be similar, as well as the unit of measurement, while the wording is different. Examples include:

• Community Level, Education #9 = Family Level, Education #8, 9, 10.

• Community Level, Education #5 = Family Level, Education #7.

Next, there are terms we have identified that would need clarified definitions before moving forward to ensure the network is collecting similar information:

• Low-income

• Community

• Living wage

• “New” in characteristics report

Lastly, the variation of percentages, numbers, and rates seems confusing and inconsistent. We understand the need to examine percentages when looking at an increase or decrease and numbers are helpful in other circumstances. The way some indicators are written it causes confusion on what is being counted or measured. The rates should most definitely be examined. However, we believe that considerations should be given to including this information in the community needs assessment to be analyzed over the years to see trends up or down.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

We consider ourselves fortunate to have a statewide data system that is a priority of the State CSBG administrators through discretionary funding. Many do not have this resource, which will be necessary for receiving an accurate unduplicated count. Training and technical assistance (T/TA) will be necessary on counting with consistent definitions and systems/procedures for efficient data collection. We also know the web based data tool in the community commons platform will become even more of an asset. The 19 Community Action Agencies in Missouri utilize this tool to collect and analyze secondary data sets for a number of purposes. The indicators that are available through this tool at the state level should be considered in relation to the proposed indicators.

Comment 18 (City Of Austin Health And Human Services ; State in TX)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

3. The quality, utility, and clarity of the information to be collected; and

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Comment 19 (Colorado Community Action Association; State Association in CO)

Change the project name/description to better reflect work being done. The work represented in this OMB FR1 is not the next generation of Results Oriented Management and Accountability (ROMA), but is the next generation of National Performance Indicators (NPI’s) and Output Reporting for our network. We are not changing the ROMA cycle, nor are the curriculum or the core elements of ROMA (assessment, planning, implementation, data collection, and analysis) changing. While better analysis is a part of the current ROMA Next Generation work scope, the bulk of the changes proposed are around reporting. Consider narrowing the name of the project so that concerns about changing NPIs do not impede agencies from continuing to incorporate ROMA principles into their work. Furthermore, HHS/ACF/OCS should address the need for true ROMA Next Generation work, which would encompass systemic changes to data management philosophies and systems locally, and would need to be supported through substantial financial and training/technical assistance resources nationally (on top of the network’s meager annual allocation of $700 million). The main point of ROMA Next Generation (as originally presented to the network), was to show the long-term impact of the CSBG program on individuals, families and communities…the changes represented in this work does not move our network closer to showing this long-term impact.

2. Extend the rollout date out by another year. While October of 2017 appears to be far into the future, 13 months is not a long time for agencies and state offices to make needed data management changes to track the demographics, services and outcomes being proposed? Given the late rollout of the ROMA Next Generation materials, this 13 month window is very small. Extending rollout by at least one year would be appreciated locally, and it would provide the anticipated new HHS/ACF/OCS leadership to provide guidance, feedback and input into what are crucial changes in how our network represents our work to the public. Additionally, this timeframe does not recognize the time needed to fully inform and educate a network the size of Community Action (33 diverse grantees in CO) about the work of ROMA Next Generation. While the work group and national partners have been working on ROMA Next Generation for three years and State Associations/RPICs and State CSBG Offices have been a part of ongoing discussions during that time, local CEEs still need time to digest this information, and to implement the necessary infrastructure to implement any required (or voluntary) changes. Colorado CEEE’s have been primarily focused over the last year preparing to meet CSBG Organizational Standards, the last large-scale national effort, and have not been as focused as the national partners and associations/state offices have been on ROMA Next Generation. Even in a State like Colorado where the State Association and State Office have been extensively publicizing and educating our grantees on the work and proposed changes included in ROMA Next Generation, true understanding of the implications of these changes are lacking, both locally and at the State level. We as a network should take the time necessary to truly define the most relevant reporting data, and then ensure that we are maximizing our efforts around those areas, whereas the changes represented in this OMB FR1 are far too wide-spread and generalized to truly show our networks impact (especially over long-term periods). While we doing this work all want the network to be successful, and we realize improving reporting is key, we also have to balance this need to move the Network forward with what a national network like ours can do efficiently and effectively. Again, with less than $700 million coming into the Network annually, we have to balance expectations with capacity (a key component of ROMA actually!). If a segment of the proposed reporting is better to be internally used, pull it out and talk about it separately as a good practice and encourage local voluntary usage.

3. Address the unique issues of Public entities that receive CSBG. Public agencies (of which 30 of the 33 CEE’s in Colorado are designated) that sub-contract most or all of their CSBG funding to other service providers may face added challenges of requiring their subcontractors to collect data in accordance with ROMA Next Generation requirements since it may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet the reporting requirements. Additionally, without significant coordination of data systems in these communities, it is extremely difficult to obtain unduplicated demographic and service counts. This challenge is not addressed in the proposed reporting package.

4. Increase and focus investments on Training and Technical Assistance. As with CSBG Organizational Standards, local Colorado CEEs, our State Association and our State CSBG Office will need training and technical assistance (and the resources and time to implement this T/TA) to increase our familiarity with the new reporting requirements and voluntary tools. Our network has very limited resources geared towards T/TA and currently, those resources do not match the scope of T/TA needs locally. The Colorado State Association and State Office have found that intensive, in the field training is well received and beneficial to local CEE’s but this takes time, preparation, engagement, and most importantly financial support. Increased T/TA investments should be made via State Associations/RPICs, national CSBG partners, State CSBG Offices, and local CEE’s. We need to continue (and increase the) training on the proposed ROMA Next Generation components, and position local CEE’s to be able to use the information contained in our tracking of NPI’s & Outputs to make management decisions, not just provide numbers to our State Office and eventually our national partners.

5. Remove Social Indicators/Population Indicators. The use of social indicators in ROMA reporting is concerning, as programs and services provided by local CEE’s do not operate in isolation and social indicators are impacted by numerous actions outside their scope and control. Inclusion of items such as community literacy rates, foreclosure rates, crime rates, high school graduation rates, emergency response time, etc. is inappropriate in the NPI’s (but is appropriate in the Community Needs Assessment/Community Action Plan as required by the CSBG Act, which is where these indicators should be discussed and prioritized as indicators of change). Even if CEE’s were able to narrowly define their “community”, this type of reporting becomes inappropriate for rolling up to the state or national levels, and collecting data from existing sources would not be accessible or at minimum, extremely challenging, and would be virtually impossible to verify at the State level and National level.

6. Remove the Baseline Characteristic Report but retain the All Characteristics Report. The reporting of the number and characteristics of customers served by CEE’s each year is important data to be collected and is required by the CSBG Act. However, the proposed Baseline Characteristic Report will require additional work for local CEE’s (and is often not possible for many CEE’s given their current data collection systems), and will not provide information that is useful or comparable at a state or national level. A report such as the proposed Baseline Characteristic Report has little value, usefulness or integrity given how local agencies operate and how customers access services. It would be a much better use of resources to develop tools to help local agencies do internal analysis rather than attempt to have an aggregate report at the state or federal level. Improved local analysis would be a significant outcome for ROMA Next Generation (but is not currently encompassed in the proposed changes).

7. Reinstate an Agency-focused goal. I believe a core mandate of Community Action is the infrastructure of local agencies located across the US to serve low-income families in rural, urban, and suburban America. We as a network need to prioritize agency improvements, and the need for federal and state investments and improvements, as well as the need for local agencies to invest in their own improvement. The CSBG Organizational Standards are an important element in setting national expectation, but setting a national goal for agency improvement is necessary to maintain focus on network-wide quality improvement over time and is in line with the purpose and intent of the CSBG Act.

8. Remove Module 1. The entirety of the Module 1 - State Administration was not a part of this process until after the initial public comment period conducted by NASCSP/OCS in early 2016, and has not been properly vetted and commented upon by State Offices, and thus should be removed from this OMB FR1 process and created through its own process (including convening a specific State Office stakeholder group review ahead of network comments). Not only are there substantial changes to the State Office “Final Report” represented in this module (which again were not properly reviewed/commented upon by the network), but the timelines involved are unrealistic in having State Offices required to provide much of the baseline information contained in this module at the end of August 2016 (in the Model State Plan), for a report that has not yet been formally adopted.

Comment 20 (Colorado Department Of Local Affairs ; Other/Unknown in CO)

l. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The information collection adds extraneous content to reports that will not be utilized by the majority of our grantees, who are only required by our state office to report on areas in which they serve. The number of reporting elements has more than tripled with no clear strategy or value to eligible entities, states or the overall CSBG network. The volume of demographic information is not practical for client intake purposes and grantees have indicated that the additional data requested of them would not be useful for other purposes within their agencies.

In addition, the level of required reporting does not take into account states with smaller allocations and/or larger numbers of grantees. Colorado receives approximately $6 million annually. In 2016, 24 of our 33 (73%) eligible entities received less than $200,000 based on the formulaic distribution, and 14 of those (58%) received less than $50,000. The proposed volume of reporting is simply not reasonable for the amount of funds received. Although we would like to inform larger projects that drive toward the impact of CSBG on poverty in our state and nationwide, it is not clear how the proposed additions will help our network achieve that goal.

2. The accuracy of the agency's estimate of the burden of the proposed collection of information Agencies estimate that the burden of proposed collection time has been grossly underestimated. In addition, there is additional time required for state staff to evaluate the information submitted for clarity, completeness and accuracy prior to submission to NASCSP, as well as the cleaning memo process for the data. We estimate this will increase current time spent on reporting oversight for eligible entities by CSBG staff by 25% or more.

Additionally, Module 1 requires extensive reporting of state agencies that was not included in the original comment period and was not given adequate time for review or consideration by state agencies. In the June 21st NASCSP webinar introducing the module for the first time, the workgroup of 6 states said they met just "2 or 3 times". This is inadequate time and consideration for this level of required change.

Because Module 1 is designed to align with the state plan, it does not make sense to include this data in ROMA Next Generation as it is not yet connected to overall services or community action strategy. See additional comments under the specific module section of this letter.

3. The quality, utility, and clarity of the information to be collected; and Public agencies who sub-contract most or all of their CSBG funding to other service providers face added challenges of requiring their grantees to collect data in accordance with ROMA Next Generation requirements. This may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet the reporting requirements, even when other more qualified service providers may exist. In rural areas, agencies may not have the capacity to implement ROMA but may fulfill the needs expressed in a community needs assessment and be fully allowable under the CSBG Act.

In Colorado, 14 of 33 eligible entities (42%) hold 42 subcontracts. Many of these entities cover large geographies in rural parts of the state. Subcontracting is an efficient way to maintain local control in the community needs assessment and program design, while complying with oversight and administration requirements for the grant. These subcontractors are invited to participate in ROMA trainings and many do attend, but mandating full implementation of ROMA for the limited funding received, for programs well within the boundaries of what is allowable for CSBG, would be excessive. If required, many high need communities in rural parts of Colorado would be excluded and would go unserved.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Most Colorado grantees report into an average of 5-8 reporting systems for funders and all of these systems are funder-driven, not client centered. In our view, this is backwards. Our state office is aggressively pursuing shared data opportunities to link systems around client data, rather than require additional reporting into another new system. We understand that HHS has an interest in interoperability systems that have been investing funds toward interoperability since at least 2011, and yet this strategy is not being considered or employed by OCS for CSBG. This is shortsighted and not forward-looking in gathering meaningful data that is sought in ROMA Next Generation.

Comment 22 (Community Action Agency ; Other/Unknown in NE)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

3. The quality, utility, and clarity of the information to be collected; and

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Comment 23 (Community Action Agency Of Beaver County ; Eligible Entity/Local Agency in PA)

As a public CAAP agency the overall requirements would be a very heavy cost burden, which we would not have the funds and also the time would be astronomical and again as a public CAAP we do not have the capacity or the cost to add staff. Our subcontractors would also be in the same situation.

The Community Action Council of Portage County (Ohio) has a relatively small CSBG grant of approximately $281,000 in FY 2016. We estimate that the implementation of ROMA Next Gen would require an investment of CSBG funding of approximately 15.4% of our total CSBG allocation, or approximately $44,000 per year, assuming no significant additional software, hardware, or related costs. This amount reflects predominantly staff time. If the current scheduled implementation date holds true, we estimate that it would require an additional initial investment as well. Our conservative estimate of this cost would approximate $12,000.

This amount above assumes that our state CSBG office (Ohio Development Services Agency) will be responsible for development of the electronic data tracking system that will be needed for reporting purposes, including bearing the costs to do so.

Community Action Committee of Victoria, Texas (CACVT) supports comments filed by the Texas Department of Housing and Community Affairs regarding proposed changes to the CSBG Annual Report.

CACVT feels staff will be required to capture such a large amount of information and to work with such a complicated reporting system that there won’t be time to actually help the people that have come to us for assistance to transition them into self-sufficiency.

Combined Community Action, Inc., Giddings, TX supports comments filed by the Texas Department of Housing and Community Affairs regarding proposed changes to the CSBG Annual Report. I sent comments last week regarding the CSBG Annual Report.

Community Action Commission of Erie, Huron, & Richland Counties Inc. has determined that the attached costs will be $53,418.31, which is 10.72% of our annual CSBG allocation of $498,495. However, there is the compounded aspect of systems compatibility that need to be factored and cannot be ascertained until the actual implementation.

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility. The proposed collection of information is necessary for proper performance of function of the agency in that it will enhance the ability to effectively quantify outcomes and measure that are not currently reflected due to some data reporting limitations in the ROMA environment.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information. CACEHR anticipates the initial burden of decreased efficiency attributed to learning curve and ability to transition into competence with the updated environment. However, after initial implementation actual burden instance is expected to level off or decr4ase because of environment proficiency.

3. The quality, utility, and clarity of the information to be collected; and The quality, utility, and clarity of the collected information will depend heavily on the new workflow of the environment. ROMA’s new updates appear to focus on providing the ability to more accurately provide pertinent detailed outcomes and measures related to the four module protocols. (i.e. Mod 1-State Administration, Mod 2-Agency Expenditure, Capacity, and Resources, Mod 3-Communtiy Level, Mod 4-Individual and Family Level)

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. This area is difficult to adequately determine due to ROMA Next Generation actual performance and data inputs not being available in conjunction by agency personnel and sister agencies, as well as lack of empirical result and outcome. More time is needed to gage additional improvements.

To Whom It May Concern:

Based on our calculations, the cost for ROMA Next Gen implementation is conservatively estimated to be $24,000 per year. We receive about $240,000 annual CSBG funding, so this represents a minimum of 10% of our budget spent tracking and reporting. We are concerned that to accomplish the required collection and maintenance of data, the proposed system would divert resources from our direct service for low-income customers.

South Plains Community Action Association supports comments filed by the Texas Department of Housing and Community Affairs regarding proposed changes to the CSBG Annual Report and we feel this unfunded mandate will spike expenses for additional staff and upgraded systems in order to capture the enormous amount of data as proposed. Local staff will be steered away from helping people to capturing a vast amount of information and navigating a complicated reporting system which may yield less impressive results of the CSBG program because staff is re-directed from direct services to paperwork. Indicators won’t necessarily capture the variety of Community Action achievements and projects and won’t tell the true CSBG story. Cost estimates are difficult to gage due to many unknowns at this point in time.

As the CEO of the Multi-Service Center, a community action agency in Washington state, I am writing to provide my feedback on the proposed changes to reporting for CSBG and the cost estimates provided by HHS related to these changes. I am very concerned about the implications of the proposed changes as well as the accuracy of estimates provided by HHS.

· The information obtained from the report that is being proposed will not have a significant increase in usefulness or being meaningful, at least not sufficient to justify the effort required to complete data collection and submission of the annual report. If the 6 NPIs and the past reporting process weren’t providing sufficiently reliable data, I struggle to see how increasing the complexity helps in this matter. More data doesn’t necessarily equate to effectiveness, especially given the concerns about capacity to collect and report data with integrity. In addition, with increased complexity comes increased possibility for ambiguity and multiple interpretations. How will CAAs in each state not to mention around the country come to mutual agreement regarding the meaning of each indicator? A shared understanding of the indicators is necessary for uniformity and for the data to be meaningful and valuable.

· I am unsure as to what methodology OCS used to arrive at the estimate regarding length of time to complete the new report. Therefore, it is difficult for me to determine if this estimate truly reflects the realities of day to day operations of a community action agency. Furthermore, I did not see any information or analysis of the existing capacity of CAAs to collect and report data with the level of complexity outlined in the proposed reporting format. Certainly it is impossible to assert that every CAA is at the same level of capacity in this regard as it relates to factors including but not limited to software/database capacity and staffing resources.

· Staff responsible for using the current report estimate that the new format will double or triple the time it currently takes in order to do a good job with the proposed process. Given that there are no additional resources being provided to offset these costs, the increased time required to complete the report will take time and resources away from providing the services that fulfill the mission of CAAs in our communities.

I feel it is critical that OCS take more time to develop shared definitions of indicators and to investigate more thoroughly whether or not CAAs currently have the capacity and resources to meet the new data requirements. At the very least, it would be helpful to know what methodology was used by OCS to create its cost estimates. It would also be helpful if OCS could advocate with other government entities to allow information to be shared or cross walked.

Ultimately, we feel strongly that data is a critical element in telling our story and evidencing the effectiveness of community action in alleviating and eradicating poverty and we appreciate OCS’ efforts to improve data collection and reporting. We simply want to ensure that the data being collected is, in fact, accurate, meaningful, and useful and that collecting and reporting data does not negatively impact service delivery and outcomes as a result of not having sufficient capacity to carry out the reporting process.

Comment 24 (Community Action Agency Of Butte County Inc. ; Other/Unknown in CA)

Thank you for conducting the process that invites comments on the Annual Report proposed as pan of the ROMA Next Generation (RNG) data collection, reporting and management system. The RNG concept has been reviewed by many interested
parties with significant input offel•ed during its' development process over the last 12 months. From the national network of field practitioners there has been both varying degrees of agreement and disagreement with various elements of the content. Because it is the nature of community action to be forthright on the actions necessary to improve the conditions and causes of poverty then some of that will be included here. At this time I also think it appropriate to make clear that I am submitting these remarks in my role as the chief executive of a local member of the community action network and not in any capacity of leadership in -that network.

We find ourselves in the midst of much change as we review this proposed method to help us to be better The evolution of language and terminology, of the manner and methods of evaluation should not be cause for us to do nothing. Rather we can use it to help us to be mindful that what we do now should bear the marks of what is enduring. I fit lends itself to varying or changing interpretations then we should lead lightly since ours is a strategy still evolving into a type of discipline. In my 38 years of work in the field of community action at the local, state, regional and national levels I have been committed to seeking excellence in our work. I remain committed to the notion that there is nothing like community action to serve as a tool for our individual and collective conscience in pursuing that desired state for our communities and those who live in those communities. It is both process and product; communities and community members working together to maximize opportunities for all.

Comments Re Next Cienu•atien Annual Report
Input to follow the four questions
I Will provide comment on the four (4) questions listed in the Dear Colleague Letter. It will not be in the format that was requested i.e. "by module, section, and page number as appropriate" because that approach does not acknowledge or provide for some very real concerns that are inherent but not obvious when you're focused on the end result i.e. the Annual Report. Certain comments reflect the continuing concern with some of the basis for the Annual Report that is not clearly evident in the modules but of definite importance when you drill down into the "how" of how the proposed Annual Report must be completed to be of a sufficient utility so that it offsets the investment of time, funds and more. The Annual Report reflects a significant revision to the type of data to be collected, the costs and methods of collection of that data and the manner in which it is used. It is my assessment that the full implications of these revisions have not been given adequate consideration such that the federal government and network of agencies would be better served to put off approval of the proposed ROMA Next Generation proposal until sufficient analysis, understanding and agreement has been achieved.

Here I address multiple points including: a) use of social indicators, b) Baseline Measures Report, c) Collective Impact measures and d) the lack of cost effectiveness of the effort to complete the necessary data collection for the Annual Report to be of sufficient off-setting utility at the local level and beyond.

Under-resourced Implementation to hinder achievement

A concern is now expressed that the design of the implementation of the proposed Annual Report does not include adequate resources to ensure its efficient and cost-effective intent. Having experienced the effort to roll-out the first generation of the ROMA system of management and reporting I can say that, from a national perspective, it was under-resourced. It took approximately 10 years (to approximately 2003) for the main points of ROMA to become well-enough known for there to be common or shared understandings amongst CAA's and the state and national systems of oversight and support. It then took an additional 10 years (to approximately 2013) for the national network to embrace the ROMA system as a management tool with the importance originally intended. The reasons for this extended period of embrace, I believe, are the inconsistent provision of resources for training and technical assistance throughout the national network.

That situation was remedied to a major extent with the current administration over the last 4 years.

However such a vision for support before, during and after implementation is absent from the proposal.

We understand that this is not the intent of the Agency. Be that as it may, without the planned resources for a roll-out, the chances are great that it will suffer the same fate, thereby negating the beliefs anticipated in its conceptualization and design.

Collection of such data is an entirely different matter calling for the development of methods of collection, entry and management. In each such case it will require that a unique definition of the data must occur. It is critical that there be common understandings and definitions agreed to, documented and shared in order for meaning to be drawn from data. But it does not stop there since then the common understanding then has to be communicated to others in different times and in different spaces without guatzntee that it will be accepted. This creates an untenable situation where data is collected without value because of the lack of shared understanding and meaning.

I believe that the agency's estimate of the burden of the proposed collection of information is understated. Local CAA's vary tremendously in there. Capacity to respond to data collection and reporting requirements. For some time now CAA’s have operated under in an environment that demands restrictions on data collection and sharing. These restrictions are inherent in grant agreements and other program support guidance. The result has been a very inefficient approach to the important aspect of data collection, management and reporting. One fix could be for OMB to impose overarching requirements on grant programs but even that would have much to overcome since the sheer amounts of required data currently being collected due to multiple grant program requirements from as many different federal agencies through a myriad of software systems is overwhelming.

The proposed Annual Report does not do anything to reduce this existing burden.

Future fixes amount to unknown costs and impacts

In the short term the NPI structure includes potential new data fields that will require new efforts to design existing software solutions. However these will be one-time efforts, with associated costs, that should not continue to impact future data collection costs. The proposed Annual Report presentations include extended timelines to accommodate the development of solutions for efficient use of technology that will be necessary for the Annual Report to be reality. While the inclusion of an extended time is warranted it is also cause for concern that the fixes envisioned and, necessary will be in the future at an unknown cost to financial and human resources. The result can easily be seen as a drain on existing and future resources intended for mission-related activities.

As to the estimate of the burden it has been my experience that most agencies in the network will involve a combination Of specialists/managers and support staff to assist in the collection and review of the data that represents their compliance and performance efforts i.e. not an unimportant task. The more proficient agencies will utilize standard data quality testing methods at identified intervals to ensure accuracy and reliability. Reconciliations and/or interpretations are generally handled by management to ensure understanding of system issues or program issues. This assumes that existing systems are adequate to the task of collecting under a new NPI structure. The nature of systems is change. Therefore the cost and effects of systems tweaking or outright change will need to be considered. In no small number of cases a significant amount of resources may be needed since NPI's require data to be collected of various sorts to be able to estimate change or accomplishment over time.

The proposed Annual Report relies upon a reporting framework that utilizes a new Theory of Change (TOC). The inclusion of a TOC is an important new development in the community action field and establish the work of the network as one worthy of consideration. However the proposed TOC excludes the critical development of a strong national organizational structure that will be the agent of change in helping to deliver relevant and catalytic activities including programming, problem measurement, new collaborative systems designs, evaluation etc.

It is the contention of many in and out of the network that the proposed TOC should include a goal for the Agency. This change would ensure that local communities focus on both the desired end of increased opportunities for all as well as the important development, maintenance and strengthening of the means by which the desired end is accomplished. The end is commonly accepted that local conditions for low-income persons be improved; opportunities maximized and low-income persons achieve improved self-sufficiency. The commitment to improvement (i.e. the means as embodied in a CSBG Eligible Entity) will not embody all of the improvement desired but it can safely be said that the end will not happen without the means.
Many local institutions exist for broad reasons in the life of a local community. In the Private sphere these institutions are primarily focused on targeted constituencies whether they are the: religious, aged, artists, youth groups, recreation groups etc. Local government exists for all local constituencies. CAA's exist to focus on the conditions and circumstances of poverty i.e. the sole purpose of the Community Services Block Grant. Any assumption that a focus on strengthening local CAA's is not needed as concept-delivery mechanisms to maintain and grow the efforts that will be required to accomplish change over time is unrealistic, Therefore it is highly recommended that the role of Agency be emphasized more by establishing a goal for it in the TOC.

In summary conclude that the field of community action is one that is ripe for continued exploration of ways to improve various aspects. It is this commitment to seeking new ways to achieve our goal of maximizing opportunity for all including low-income residents that is a driving force in conducting pilot projects, pursuing trends in relevant evaluation and, where possible, scalability. To varying degrees CEE's have been a community-based laboratory of Service-delivery and innovative approaches for many years. That is why did not respond to the three questions in the. Federal Register notice regarding the data elements. That discussion should be the subject of subsequent efforts to ensure the maximum amount of efficiency is designed into the collection of key data.

The proposed Annual Report, while commendable in concept, requires too much work to get it to the point where its cost-effectiveness warrants implementation because of the benefit that it poses to the problem. The problem is how to improve the reporting of a national network that will enable greater understanding of the impact that community action is achieving as it has since its inception in 1964. The proposed Annual Report is not the solution to that problem. It is a valiant attempt at addressing important issues that deserve attention but its overreaches for solutions poses more challenges than the problem. Some of the challenges may be overcome, in the course of time, with appropriate approaches to problem-solving.

To reiterate that the capability is present the national network of CEE's is glad to have been integral participants in the companion eff01ts to improve accountability via the creation and implementation of national Organizational Standards for all local CSBG Eligible Entities. We are also glad to have been an integral part of the effort to create State and Federal Accountability Measures since these then address the full complement of responsible parties to the unique mission that is community action.

It is my sincere hope that these comments are taken with the intended purpose; to be candid comments on a concept with far-reaching implications. It is also my hope that what is clear through these comments is that I share a vision of the community action network that will always be characterized by the desire and commitment for the work that we do at all levels to be viewed as excellent, valuable, accountable and a valid expression of this country's commitment to promoting opportunity for all who call it home. Thank you, again, for this opportunity to share my perspectives on this very important issue with you.

Comment 26 (Community Action Agency Of Northwest Alabama, Inc.; Eligible Entity/Local Agency in AL)

Views represent the Agency Executive Director and staff following an Agency staff meeting August 5, 2016. The HHS table estimating the hours needed, on average, is below. According to HHS they are still working on associating dollars with this table. Number of Respondents 52 Grantees 1,035 Sub-Grantees Number of responses per respondent 1 Average burden hours per response 164 242 Total burden hours CSBG Annual Report 8,551 250,585 Items to include in cost estimate of yearly, post-adoption ROMA NG work TIME 1. The amount of time that intake and other workers will spend obtaining information that is not required by other programs. Given the expansive ROMA NG measures, we estimate the documentation/follow-up time to be double per worker. Currently a worker has 2,080 work hours per year. While this number will not change, what shall change is the number of individuals and families assisted. Excluding LIHEAP we currently work with 900-1,000 households each year overall. ROMA NG will cut in half the number of emergency needs that can be addressed due to the burden of over 1,000 new measures and follow-up on each one.

2. The typical amount of time required for each worker to be trained and refreshed on the skills of data entry, or data acquisition, including new and continuing staff. Initial training will be extensive. First the certified ROMA trainers will need to be trained along with the executive director, followed by the staff and board. Software training will be a crucial step for the responsible staff to understand the changes and to train agency-wide staff. ROMA Trainers time is estimated at 80 hours each for two Trainers = 160 hours + 8 hours training staff + 4 hours with Board = 172 hours. Executive Director = 40 hours. Software staff = 120 hours + 104 hours in training/tech support for staff. All other staff = 20 hours x 13 staff = 260 hours + additional hours individual tech support. 8 hours x 13 staff = 104

Total Agency time in initial change = 800 hours or more (if no new employees are hired) 3. The amount of time your team will spend on data quality assurance, as well as quality assurance on the final report. In the beginning reports will need to be looked at weekly with the ROMA Trainers and ED then reported to staff where we are failing to track data. Following this, monthly reports at staff meeting and monitoring by lead staff will need to take place to prevent falling behind. Final reporting will be time consuming to ensure all data is correct and contact employees to make corrections as necessary. Executive Director/ROMA Trainers weekly monitoring/staff support = 10 hours. Staff support = 3 hours weekly. Final reporting = 12 hours Total quality assurance = 25 hours 4. The amount of time staff will spend tracking individuals and families, tracking progress of community projects and recording the progress in the format required in modules 3 and 4. Our Agency provides services to 900-1,000 households each year not including LIHEAP. It is simply unrealistic to assume current staff can track this number for 180 days after a service is provided. Answering the ‘measures’ may force the Agency to eliminate some clients believed to not meet the level we seek for achieving outcomes. 1,040 hours year x 8 staff = 8,320 hours 5. The amount of time staff will spend gathering external demographic information required in module 4. This will mainly be Executive Director and Admin Assistant who are responsible for final reporting. 80 hours year 6. The amount of time spent tracking volunteers, board members and their commitments to organizations and projects that are not run by your Community Action Agency. This simply cannot be tracked. We would not even try. 7. The new reports to the state required for each of the 50 organizational standards, with additional analysis required on any standards that your agency has not achieved in full [module 1.] More reporting time = 8 hours 8. The placement of data into the state reporting software, including checking and answers to the narratives required by the state for module 1, as well as the local information on funds, people and projects. After reviewing the format suggested for State use it will be time consuming to hand enter this information unless the state shares the on-line tool. Time to collect information will need to be on-going yearly = 40 hours for reporter Writing narratives = 40-60 hours yearly 9. The Cost of Time = in excess of $150,000. This change would roughly entail ¼ of the Agency current staff time x salary for Agency staff. O Based on the full cost of the different levels of staff contribution time.

Comment 27 (Community Action Agency Of Northwest Alabama, Inc.; Eligible Entity/Local Agency in AL)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility Certainly not all indicators are practical for every community action. Many of the proposed indicators are totally unrealistic for a small agency and in the end will only be a number not having effect on the daily operations. The levels of information to be collected in many areas cannot in good faith be obtained creating the illusion of failure. ROMA NG disregards the Standards which are so rigid and control much of the agency activities. The Standards ensure agencies must provide/produce proof of actual documentation of activities. With this documentation comes the assurance work is being done in the community and for the agency. It is not necessary to add multiple layers using many documents to this already cumbersome process. Perhaps the Standards and ROMA NG should have been rolled into one process and streamlined for reporting.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information. What is being proposed (242 hours per response) is considerably less than our estimates which would in many ways equate to almost ½ of a staff time for training, collection and reporting.

3. The quality, utility, and clarity of the information to be collected; and It is unrealistic to think staff feeling the constant pressure to perform would gather quality data. When it becomes more about ‘meeting an indicator’ than meeting a client need the data gathering or long-term reporting could suffer. We are expected to report on all funding resources/revenue regardless of having administration funds to do so. Many of our resources have no admin at all. We try to work effectively and efficiently within the funding sources guidelines to help our community. To be required to add additional reporting of data collected is non-productive.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. Consider what useful information for reporting purposes is. If it’s only use is on the local level why collect this? It would certainly be difficult to aggregate some of the data into useful state-wide report s.

Comment 28 (Community Action Agency Of Northwest Alabama, Inc. ; Eligible Entity/Local Agency in AL)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility Certainly not all indicators are practical for every community action. Many of the proposed indicators are totally unrealistic for a small agency and in the end will only be a number not having effect on the daily operations. The levels of information to be collected in many areas cannot in good faith be obtained creating the illusion of failure. ROMA NG disregards the Standards which are so rigid and control much of the agency activities. The Standards ensure agencies must provide/produce proof of actual documentation of activities. With this documentation comes the assurance work is being done in the community and for the agency. It is not necessary to add multiple layers using many documents to this already cumbersome process. Perhaps the Standards and ROMA NG should have been rolled into one process and streamlined for reporting.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information. What is being proposed (242 hours per response) is considerably less than our estimates which would in many ways equate to almost ½ of a staff time for training, collection and reporting.

3. The quality, utility, and clarity of the information to be collected; and It is unrealistic to think staff feeling the constant pressure to perform would gather quality data. When it becomes more about ‘meeting an indicator’ than meeting a client need the data gathering or long-term reporting could suffer. We are expected to report on all funding resources/revenue regardless of having administration funds to do so. Many of our resources have no admin at all. We try to work effectively and efficiently within the funding sources guidelines to help our community. To be required to add additional reporting of data collected is non-productive.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. Consider what useful information for reporting purposes is. If it’s only use is on the local level why collect this? It would certainly be difficult to aggregate some of the data into useful state-wide report s.

Items to include in cost estimate of yearly, post-adoption ROMA NG work TIME 1. The amount of time that intake and other workers will spend obtaining information that is not required by other programs. Given the expansive ROMA NG measures, we estimate the documentation/follow-up time to be double per worker. Currently a worker has 2,080 work hours per year. While this number will not change, what shall change is the number of individuals and families assisted. Excluding LIHEAP we currently work with 900-1,000 households each year overall. ROMA NG will cut in half the number of emergency needs that can be addressed due to the burden of over 1,000 new measures and follow-up on each one.

2. The typical amount of time required for each worker to be trained and refreshed on the skills of data entry, or data acquisition, including new and continuing staff. Initial training will be extensive. First the certified ROMA trainers will need to be trained along with the executive director, followed by the staff and board. Software training will be a crucial step for the responsible staff to understand the changes and to train agency-wide staff. ROMA Trainers time is estimated at 80 hours each for two Trainers = 160 hours + 8 hours training staff + 4 hours with Board = 172 hours. Executive Director = 40 hours. Software staff = 120 hours + 104 hours in training/tech support for staff. All other staff = 20 hours x 13 staff = 260 hours + additional hours individual tech support. 8 hours x 13 staff = 104

Total Agency time in initial change = 800 hours or more (if no new employees are hired) 3. The amount of time your team will spend on data quality assurance, as well as quality assurance on the final report. In the beginning reports will need to be looked at weekly with the ROMA Trainers and ED then reported to staff where we are failing to track data. Following this, monthly reports at staff meeting and monitoring by lead staff will need to take place to prevent falling behind. Final reporting will be time consuming to ensure all data is correct and contact employees to make corrections as necessary. Executive Director/ROMA Trainers weekly monitoring/staff support = 10 hours. Staff support = 3 hours weekly. Final reporting = 12 hours Total quality assurance = 25 hours 4. The amount of time staff will spend tracking individuals and families, tracking progress of community projects and recording the progress in the format required in modules 3 and 4. Our Agency provides services to 900-1,000 households each year not including LIHEAP. It is simply unrealistic to assume current staff can track this number for 180 days after a service is provided. Answering the ‘measures’ may force the Agency to eliminate some clients believed to not meet the level we seek for achieving outcomes. 1,040 hours year x 8 staff = 8,320 hours 5. The amount of time staff will spend gathering external demographic information required in module 4. This will mainly be Executive Director and Admin Assistant who are responsible for final reporting. 80 hours year 6. The amount of time spent tracking volunteers, board members and their commitments to organizations and projects that are not run by your Community Action Agency. This simply cannot be tracked. We would not even try. 7. The new reports to the state required for each of the 50 organizational standards, with additional analysis required on any standards that your agency has not achieved in full [module 1.] More reporting time = 8 hours 8. The placement of data into the state reporting software, including checking and answers to the narratives required by the state for module 1, as well as the local information on funds, people and projects. After reviewing the format suggested for State use it will be time consuming to hand enter this information unless the state shares the on-line tool. Time to collect information will need to be on-going yearly = 40 hours for reporter Writing narratives = 40-60 hours yearly 9. The Cost of Time = in excess of $150,000. This change would roughly entail ¼ of the Agency current staff time x salary for Agency staff. O Based on the full cost of the different levels of staff contribution time.

Comment 29 (Community Action Agency Of Southern New Mexico; State Association in NM)

OCS REQUESTED FEEDBACK:
1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility. Pg. 25. Module 2. Section B Number 1. I don’t see a reason or value in separating volunteer time by type of activities. I believe most of the information can have practical utility.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information- Currently our agency has 8 people involved in the NASCSP yearly report process that is over 300 hours in total. This doesn’t include time during the year collecting data. The new information will significantly increase the number of hours we spent on NASCSP during report time and during the year especially when trying to track past clients.

3. The quality, utility, and clarity of the information to be collected; and- in a perfect world the information requested is wonderful, in the real world I am not sure that the agencies will be able to gather all the information. I just don’t see how we are going to gather all the information. Especially the Education and Cognitive areas.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. We need the information in the same form as the state program managers have. we have excel sheets that we currently use and they are outdated

Comment 30 (Community Action Association ; State Association in NM)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility. Pg. 25. Module 2. Section B Number 1. I don’t see a reason or value in separating volunteer time by type of activities. I believe most of the information can have practical utility.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information- Currently our agency has 8 people involved in the NASCSP yearly report process that is over 300 hours in total. This doesn’t include time during the year collecting data. The new information will significantly increase the number of hours we spent on NASCSP during report time and during the year especially when trying to track past clients.

3. The quality, utility, and clarity of the information to be collected; and- in a perfect world the information requested is wonderful, in the real world I am not sure that the agencies will be able to gather all the information. I just don’t see how we are going to gather all the information. Especially the Education and Cognitive areas.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. We need the information in the same form as the state program managers have. we have excel sheets that we currently use and they are outdated

Comment 31 (Community Action Commission Of Erie, Huron, & Richland Counties ; Eligible Entity/Local Agency in N/A)

Community Action Commission of Erie, Huron, & Richland Counties Inc. has determined that the attached costs will be $53,418.31, which is 10.72% of our annual CSBG allocation of $498,495. However, there is the compounded aspect of systems compatibility that need to be factored and cannot be ascertained until the actual implementation.

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility. The proposed collection of information is necessary for proper performance of function of the agency in that it will enhance the ability to effectively quantify outcomes and measure that are not currently reflected due to some data reporting limitations in the ROMA environment.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information. CACEHR anticipates the initial burden of decreased efficiency attributed to learning curve and ability to transition into competence with the updated environment. However, after initial implementation actual burden instance is expected to level off or decr4ase because of environment proficiency.

3. The quality, utility, and clarity of the information to be collected; and The quality, utility, and clarity of the collected information will depend heavily on the new workflow of the environment. ROMA’s new updates appear to focus on providing the ability to more accurately provide pertinent detailed outcomes and measures related to the four module protocols. (i.e. Mod 1-State Administration, Mod 2-Agency Expenditure, Capacity, and Resources, Mod 3-Communtiy Level, Mod 4-Individual and Family Level)

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. This area is difficult to adequately determine due to ROMA Next Generation actual performance and data inputs not being available in conjunction by agency personnel and sister agencies, as well as lack of empirical result and outcome. More time is needed to gage additional improvements.

Comment 33 (Community Action Council ; Eligible Entity/Local Agency in KY)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The Council is in favor of creating a universal data reporting system that accurately demonstrates the outcomes of the work conducted by CAAs. The Network would benefit from a comprehensive data system that could show the true impact of the services CAAs provide to local communities, and accurate, reliable data is the best way to accomplish this outcome. However, the proposed reporting structure is not feasible as existing funding is not sufficient to allow for a complete overhaul of each CAA’s reporting system. The proposed method of data collection will require each CAA to develop a new database system—and hire additional front line and management staff—to ensure data accuracy. If these elements are not in place, the information that is reported will be inaccurate and will not reflect the true outcomes of the CAAs and the Network.

O The Council agrees with the Partnership in that the package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

O 1.1 – The Council agrees with the Partnership in that State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance, State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

O 1.2 – The Council agrees with the Partnership in that, in addition, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple reporting systems. In addition, as many local agencies serve families over time, with no definition of “new” being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

o Concerns

1. (2.1) – The Council agrees with the Partnership in that the current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

2. (2.2) – The Council agrees with the Partnership in that, to comply with the draft, CSBG Annual Report local agencies will need to modify existing systems.

3. (2.3) – The Council agrees with the Partnership in that many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple reporting systems which adds to the time necessary to provide the data.

> Option selected describing what the Council will need to report the requested ROMA Data:

1. The agencies’ estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under estimated. Based on our calculations, the burden to our local agency would be 1440 hours. The cost of this time burden is estimated to be $21,000.00.

> The Council agrees with the Partnership in that sub-grantees will need systems that track customers, services, and outcomes, and, given the requirement of many funding sources, additional and sometime separate data collection systems are required. State grantees do not have the funding or capacity, for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden outlined here. Staff will need to be trained on the new online system.

3. The quality, utility, and clarity of the information to be collected; and

o Module 3, Section A: The Council agrees with the Partnership in that, first, Community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Secondly, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

It would be very beneficial to local CAAs to have a universal system that is created and maintained at the national level. In the current proposal, however, only states will have access to the national database. As States simply compile the information provided by local CAAs, data monitoring under the current proposal will be less efficient. If local CAAs were given access to the database, they could perform data entry, reducing the burden on States and improving data accuracy and efficiency. Local CAA access to a universal database system will streamline the process by ensuring that all CAAs are using the same format and data collection methods.

O The Council agrees with the Partnership in that sub-grantees will still need systems that track customers, services, outcomes, and, given the requirement of many funding sources, requires additional and sometime separate data collection systems. The Council further agrees with the Partnership in that new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

o The Council agrees with the Partnership in that DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS. Data systems need to be streamlined across all DHHS-funded programs.

Comment 34 (Community Action Council Of Lexington-Fayette, Bourbon, Harrison And Nicholas Counties ; Eligible Entity/Local Agency in KY)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The Council is in favor of creating a universal data reporting system that accurately demonstrates the outcomes of the work conducted by CAAs. The Network would benefit from a comprehensive data system that could show the true impact of the services CAAs provide to local communities, and accurate, reliable data is the best way to accomplish this outcome. However, the proposed reporting structure is not feasible as existing funding is not sufficient to allow for a complete overhaul of each CAA’s reporting system. The proposed method of data collection will require each CAA to develop a new database system—and hire additional front line and management staff—to ensure data accuracy. If these elements are not in place, the information that is reported will be inaccurate and will not reflect the true outcomes of the CAAs and the Network.

O The Council agrees with the Partnership in that the package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

O 1.1 – The Council agrees with the Partnership in that State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance, State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

O 1.2 – The Council agrees with the Partnership in that, in addition, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple reporting systems. In addition, as many local agencies serve families over time, with no definition of “new” being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

o Concerns

1. (2.1) – The Council agrees with the Partnership in that the current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

2. (2.2) – The Council agrees with the Partnership in that, to comply with the draft, CSBG Annual Report local agencies will need to modify existing systems.

3. (2.3) – The Council agrees with the Partnership in that many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple reporting systems which add to the time necessary to provide the data.

> Option selected describing what the Council will need to report the requested ROMA Data:

1. The agencies’ estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under estimated. Based on our calculations, the burden to our local agency would be 1440 hours. The cost of this time burden is estimated to be $21,000.00.

> The Council agrees with the Partnership in that sub-grantees will need systems that track customers, services, and outcomes, and, given the requirement of many funding sources, additional and sometime separate data collection systems are required. State grantees do not have the funding or capacity, for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden outlined here. Staff will need to be trained on the new online system.

3. The quality, utility, and clarity of the information to be collected; and

o Module 3, Section A: The Council agrees with the Partnership in that, first, Community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Secondly, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

It would be very beneficial to local CAAs to have a universal system that is created and maintained at the national level. In the current proposal, however, only states will have access to the national database. As States simply compile the information provided by local CAAs, data monitoring under the current proposal will be less efficient. If local CAAs were given access to the database, they could perform data entry, reducing the burden on States and improving data accuracy and efficiency. Local CAA access to a universal database system will streamline the process by ensuring that all CAAs are using the same format and data collection methods.

O The Council agrees with the Partnership in that sub-grantees will still need systems that track customers, services, outcomes, and, given the requirement of many funding sources, requires additional and sometime separate data collection systems. The Council further agrees with the Partnership in that new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

o The Council agrees with the Partnership in that DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS. Data systems need to be streamlined across all DHHS-funded programs.

Comment 36 (Community Action Partnership ; National Partner in DC)

This letter provides detailed comments and recommendations based on the format requested in the Federal Register. Overarching concerns with the currently proposed CSBG Annual Report include:

· Reporting requirements that exceed agency capacity for data collection, integration, and analysis.

Many agencies struggle under the burden of collecting and reporting data for multiple programs that use different – and often incompatible – software and systems. The CSBG Annual Report as currently proposed is unduly burdensome and underestimates the time required at the local level to collect, integrate, and report data at the agency level.

· Reporting requirements that produce data of limited utility for CAAs and State CSBG Lead

Agencies. The Partnership strongly believes that all data collected for the CSBG Annual Report should have a clear purpose and use for local agencies and State CSBG Lead Agencies. Caution must be taken if data reporting is included that is not intended to be aggregated and used at the state and federal level.

· Reporting requirements that would be better characterized as “best practices”, voluntarily adopted, and supported with additional training and technical assistance. As noted in our April 2016 submission, investing in training and technical assistance to increase the analysis of data or to implement focused community-level initiatives using developing formats (e.g. Collective Impact) would be a much wiser use of limited CSBG resources than requiring the reporting as proposed.

· Concerns over allocation of scarce resources are not adequately addressed. The single greatest barrier to improved data collection and analysis is under-resourced and fragmented management information systems at the local agency level. The investment of limited resources in the creation of new state-level systems fails to correct the problem as its source and may directly trade off with improvements that could be gained at the agency level through coordinated support of capacity building efforts.

Comment 37 (Community Action Partnership Of Kern ; Other/Unknown in CA)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility Collection of data by individual programs is necessary to evaluate the success of specific program services and often required to justify funding for the program. In many cases, individual programs are required to collect specific data through specific data collection process and platforms. Sharing of data is not always feasible or even allowable depending on the program and federal parent agency. Collectively gathering of data from multiple programs, with the goal of providing unduplicated service counts is appealing, but not always possible.
Our agency has performed service distribution at a high level for a long time under the current and previous reporting systems. We recognize the importance of providing unduplicated counts and family level measurement of movement toward self-sustainability when possible. Requiring unduplicated counts across all service types and through the combination of data platforms is not always practical. As an example, linking services provided to a family by CAPK VITA (Volunteer Income Tax Assistance), CAPK Youth Services and the CAPK Food Bank will require data from each user. Under the proposed reporting system, the Food Bank will be required to obtain at least 9 pieces of personal information during emergency food distribution efforts. In any case, our Food Bank will be unable to report approximately half of the food distribution customer base because they are served by private food pantries where we are not involved in direct distribution activity. If we are placed in a position of requiring pantry partners to obtain this additional demographic information from each client, we will lose many of our pantry sites. The accompanying backlash and bad publicity would not look good. We think this type of cascade effect could end up creating a situation we are trying to avoid by implementing strict reporting standards. Other question 1 responses:

· Module 1: State Agencies should be able to report the number of local agencies that have met 100%, 70%, and less than 50% of the organization standards, not just 100%. · Module 4: Remove the Characteristics for NEW Individuals and Households Report many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems.

· Module 3: Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable.

· Module 4: Propose a suggested stability indicator scale where appropriate and tailored to meet regional needs. A one size fits rule does not apply in this case.

· Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Make it clear to network agencies that improvements are expected to take place and CSBG dollars may be used to meet the needs of ROMA Next Generation reporting and tracking requirements. Recognize the potential for short term decreases in service at the expense of making the agency improvements.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

If CAPK continues partial funding of several programs with CSBG dollars, reporting and tracking requirements associated with the new ROMA standards will increase dramatically. Tracking clients after they receive services is a large part of the reason for this increase. All CSBG funded programs will require significant additional training prior to compliance with new tracking and reporting requirements. The time spent on this translates into the equivalent of more than a full time position spread over those programs. A wide variety of employees are involved with reporting and tracking. The OCS estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under estimated. Based on our calculations, the burden to our local agency would be closer to 2000 hours. The cost of this time burden is estimated to be $75,000. An agency of our size and complexity may be forced to severely limit CSBG funding to one or two very specific programs that are easy to report and track, and do not involve proprietary data concerns.

· The current IS survey includes 300+ data elements to report. The proposed CSBG

Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.
· Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, CAPK is required to use multiple (and sometimes incompatible) reporting systems which significantly adds to the time necessary to provide the data.

· The OLDC or online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems.

3. The quality, utility, and clarity of the information to be collected; and

We agree that unduplicated counts and tracking services are critical to determining the effectiveness of agency services when conducting an analysis of movement out of poverty. Reported data will only be as good as the ability of programs to share the data. If programs cannot readily share data, multiple redundant data input systems will be required leading to additional time spent (see costs associated with #2 above).

The additional reporting and tracking requirements and heavy emphasis on specific (guaranteed?) outcomes, may lead to a significant decrease in the quantity of services provided to our community. In order to prove that our quality of services will bring a family out of poverty, the quantity of services provided to the public will decrease. If this is the goal, please spell it out and make it clear so there is no confusion in the future when agencies report back with far fewer client contacts and drastic drops in some service areas. Take for instance the estimated 90,000 or so unduplicated users of the Food Bank. If our pantry partners do not agree to new data collection, we could lose half of the people that traditionally obtained CAPK Food Bank resources. We would be in a position of reporting half as many clients receiving services. How will that look in the short term? CAPK may be able to regain many of those lost clients through expansion of our own distribution system, but that could take years. The push for data on emergency food distribution could backfire for larger agencies like CAPK and end up making the State, Federal and network CSBG program look bad.

Everything about ROMA leads us to believe the goal is to show how we bring people and families out of poverty. This will require a much more targeted approach uniting all programs (ROMA Next Generation), as opposed to past business as usual (silo services). If this is truly the intent of this process, recognize the anticipated short term decreases in service that are unavoidable for larger agencies trying to meet these requirements.

Other question 3 responses:

· Several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

· Comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading.

· Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry

Form should be removed. The large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

· Module 3, Section A: Any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

We already know that several major Federal programs do not allow easy sharing of data for reporting outside of their individual program. Since CAPK deals in many program services funded by many Federal agencies (e.g. USDA, USHHS, USDOE, IRS), developing a data system capable of collecting required data is not only cost redundant, but cost ineffective. We are exploring the system requirements to meet the reporting and tracking need and estimate it could cost as much as $30,000. This is on top of the approximate $90,000 currently needed to maintain existing reporting software and associated hardware.

One significant burden to larger agencies is the need to provide full unduplicated reporting and tracking of services for programs receiving even $1 of CSBG funding. This requirement will effectively direct CSBG funding into one or two programs, removing the flexibility of CSBG that is so special to providers. The CAPK Food Bank is heavily funded by CSBG dollars and will not be able to comply with the proposed reporting and tracking standards without massive short term loses in clients and partner organizations. CAPK WIC may only receive a small amount of CSBG funding during a year, but meeting reporting and tracking issues create a burden. Consider a start point for funding levels associated with suggested reporting and tracking, maybe when a program exceeds 49% CSBG funding.

· To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 38 (Community Action Partnership Of Orange County; Eligible Entity/Local Agency in CA)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

o The practical utility of the information is unknown at this time. Much of the data is collected for other funding sources utilizing reporting forms and systems mandated by those funders. The rollout for Community Action Agencies should be on a pilot basis that includes representation of large and small CAAs, urban & rural, public & private, etc.: rather than requiring the full implementation by the whole network. We simply don’t know how well ROMA NG will work. Full implementation will lead to constant disagreement and tinkering nationwide in an attempt to implement changes and corrections (meaning lots of time & money).

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

The agency’s’ estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under estimated. Based on our calculations, the burden to our local agency would be 3 FTE Staff (approximately 6,000 hours).The cost of this time burden is estimated to be approximately $150,000. The estimate is based on a presumption that 3 data/evaluation coordinators will be hired; 1 for each of the agency’s 3 operations departments. Estimate for development of a computer program for centralized intake and coordinated reporting is for a one time purchase from $60,000 to $80,000 (plus unspecified ongoing annual maintenance costs). The centralized intake form will be in addition to the actual application form required by specific programs. This will be necessary to reliably collate data from 16 major programs in addition to CSBG. Most of these programs have their own unique reporting systems which are not easily transferable or directly compatible with the CSBG reporting.

3. The quality, utility, and clarity of the information to be collected; and

o We anticipate differences in definitions from program to program which may complicate the reporting process.

O The information/data for the proposed report is technical and complicated. Initially, it would be better to simplify reporting and systems rather than adding complicated layers and categories. As better understanding and acceptance occurs, we can improve reporting accuracy and sophistication, over time, helping to ensure competency and compliance throughout the network.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Decrease and simplify the number of proposed reporting elements. We will attempt to be responsive with information for all programs however; we do not currently collect all the data necessary to complete all categories identified for reporting.

Develop an efficient and cost effective way to report and utilize data already collected for other federal and standardized programs in their current format.

Comment 39 (Community Action Partnership Of Riverside County; Eligible Entity/Local Agency in CA)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

See comments below.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

Estimated Average Burden Hours for the CSBG Annual Report

For our local Community Action Agency, the following is an estimate for the Average Burden Hours incurred in collection, data input, data transfer, and quality assurance for the CSBG Annual Report:

• Data collection and input from intakes, applications and demographic 257 hours info. Forms

• Transfer of data from other systems (e.g. ServTraq) into current reports 54 hours for aggregation

• Aggregation of program/other agency data and demographic reports for 18 hours

CSBG Annual Report

• Data quality assurance (internal monthly reports and CSBG Annual Report) 15 hours

Total 344 hours

3. The quality, utility, and clarity of the information to be collected (see comments below)

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Our agency is currently in the process of purchasing an agency-wide case/data management system.

Comment 41 (Community Action Planning Council Of Jefferson County, Inc.; Eligible Entity/Local Agency in NY)

Thank you for allowing our Community Action Network to provide comments on the proposed OMB PRA Clearance FRN #1. We applaud you for striving to improve our performance management system.

Community Action Planning Council of Jefferson County, NY (CAPC) has reviewed the draft documents for ROMA Next Generation and the CSBG Annual Report and would like to make the following comments. We believe the progress made to date since the January 2011 is significant, and we want to keep moving forward. We want to get the development work behind us so we can move forward together with the implementation of ROMA Next Generation. In general, the requirements as written do not lend themselves to value at all levels; they are not feasible given limited financial resources, technological capacity, diversity of the Network (small/large, public/private, urban/rural/suburban), and limited T/TA fund. Our agency feels that requirements will place a significant undue burden on us as a local agency in staff time and costs associated with reporting.

Please find below our detailed comments:

Our concern, as well as our entire networks, is focused in five core areas including:

• Module 1: Reporting on CSBG Organizational Standards and Technical Assistance Plans/Quality Improvement Plans for local agencies with unmet standards.

• Module 3: Community Outcome Indicators and Collective Impact

• Module 4: Characteristics for NEW Individuals and Households

• Module 4: Stability Measures/Indicators

• Modules 3 and 4-ROMA Goals: Lack of inclusion of an Agency Goal

Question 1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

1. Module 1: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported and creates a misuse of the Organizational Standards. Expecting constant perfection is unrealistic and does not have practical utility for local agencies or the State CSBG Lead Agency. As proposed, the single metric the State Agencies will report on will be the number of eligible entities that meet 100% of the CSBG Organizational Standards. To increase utility of the information, State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

2. Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency. This is not necessary for agency performance, and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will be skewed by large population centers and decrease the practical utility of the data even further. In addition, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems. In addition, as many local agencies serve families over time, with no definition of “new” being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years (e.g. Early Head Start, child care, after school programs, youth programs, job training, WIC, food boxes, LIHEAP, etc.).

3. Module 3: Remove the Community Outcome Indicators that include percent or rates. This inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

4. Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.

5. Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the eligible entity’s work. One option duplicates data already collected and the second uses a tool that defines financial stability based on a construct the uses assumptions drawn from the general population, not low-income individuals and families specifically. Because financial well- being means something very different for the low-income population in comparison to the general population, use of the scale will produce data of limited utility.

6. Address the unique issues of Public entities that receive CSBG. Public agencies (sub-grantees) that sub-contract most or all of their CSBG funding to other service providers may face added challenges of requiring their grantees to collect data in accordance with ROMA Next Generation requirements since it may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet the reporting requirements. This challenge is not addressed in the proposed reporting package.

7. Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this

national goal that addressed the critical role the network of local agencies serving 99% of US counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network for ongoing training, technical assistance, and general capacity- building supports.

Question 2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.

Community Action is very concerned about the additional staff time and training for the data collection proposed.

• Concerns include:

1. The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.


2. The burden on local agencies is impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems.

3. Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data. Our agency, CAPC, has recently signed a contract with a Management Information for-profit company that we felt had the capacity to help us report outcomes and demographics for our programs. We have been conducting research for 8 years to find the closest match in reporting in our programs and to raise funds to purchase any system. If we are required to meet the proposed requirements as they have been written and need to include additional/revised tracking methods, it will add to the already significant cost of a new data system; and add to the burden of staff to ensure the correct information is reported.

4. The proposed estimated time burden of 242 hours per local Community Action Agency is profoundly under estimated. Based on our calculations, the burden to CAPC would be at minimum, 1,280 hours. The cost of this time burden is estimated to be $26,880. This calculation is based on an annual minimum of 80 hours per each program plus 160 hours Administrative costs. This does not include any time spent with our new software company.

• In addition, it is not clear how the burden for local agencies is decreased by the online data collection system described in the by OCS. OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden outlined here.


Question 3: The quality, utility, and clarity of the information to be collected

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

• Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading.

• Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

• Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

As noted above, OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal

DHHS Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

• OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 42 (Community Action Program Associations For Community Action Agencies; State Association in WI)

We, the undersigned, represent the state Community Action Program Associations for Community Action Agencies in Region V; encompassing the states of Illinois, Indiana, Michigan, Minnesota, Ohio and Wisconsin.

We are writing to lend our collective voice representing Region V in support of the comments submitted to you by the Community Action Partnership regarding the current OMB Clearance Package relating to ROMA: Next Generation and the draft Community Services Block Grant (CSBG) Annual Report.

Many of us submitted individual comments in the first comment period on the proposed new rules and requirements for ROMA: Next Generation and most of us will be submitting our individual agency and/or association comments on the proposed CSBG Annual Report during this current comment period. But we also wanted to comment, with one voice, on the over-arching principles and direction of the documents, as spelled out by the Partnership.

All of us believe strongly in accountability; the value of what the Community Action network provides to low-income households and the communities in which they reside can only be enhanced by transparency in activities and accountability in outcomes. But the measures used to define accountability, while they should be strong, need to also be developed and implemented in the context of the uniquely local and flexible nature of Community Action Agencies and in the realization that escaping poverty is a complex task, unique to each individual striving for economic self-sufficiency.

We believe the progress made to date (since January 2011) in the development of ROMA: Next Generation is significant and we want to keep moving forward. In doing so, we must be sure the final product has value at all levels; is realistic given existing financial resources, technological capacity, diversity of the network, and is sensitive to the burdens placed on states and local agencies.

The Partnership’s comments are consistent with these beliefs and were developed and refined through arduous effort and substantial input from the network of the nation’s Community Action Agencies, including those of us in Region V. We, thus, support them with great confidence and urge that their conclusions be incorporated wherever possible into the final rules.

Comment 43 (Community Renewal Team ; Eligible Entity/Local Agency in CT)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

As a community action agency leader, there are several areas detailed in the following sections where the collection of information proposed does not meet what we need to collect and has little to no usefulness. The proposed package is overly burdensome and is requiring the agency to collect information outside the reach of CSBG and data that we need to conduct our business operations.

We share your commitment in cultivating improved performance management. In addition we also agree that state and national reports must show the impact of the CSBG network and demonstrate results as well.

We propose the changes as follows:

a. Module 1: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses.

There are significant challenges with the way this information is being reported and potential creates a misuse of the Organizational Standards. As proposed, the metric the State Agencies will report on will be the number of eligible entities that meet 100% of the CSBG Organizational Standards. State agencies should have the option to report the number of agencies at a variety of thresholds (100%, 90%, 80%, 70%, 60 %...) that show strong performance or the training and technical needs of the agencies. Presenting this type of information in this way will reflect a more accurate picture of agency performance will allow for better tracking over time and assist the State agencies in more efficient investment of training and technical assistance.

b. Module 4: Remove the Characteristics for NEW Individuals and Households Report.

There is not practical use at the local level given the time and expense to create it at our agency. This information is not necessary for agency performance and not reflective of how local agencies and states look at their communities to assess needs, report progress and develop new programming. In addition, we serve families over time. With no definition of “new” being provided for review, it will be difficult to implement this report consistently. It becomes even more challenging when individuals or families received different services over many years (Early Head Start, Head Start, Youth Programs, LIHEAP, etc.)

c. Module 3: Remove the Community Outcome Indicators that include percent or rates.

Inclusion of the rates in the Community Outcome indicators is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

d. Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work. Giving priority to one approach when there are other options for doing community level work, is not necessary. We recommend removing this and including it in the training and technical assistance section.

e. Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of an agency’s work. The first option duplicates data already being collected. The second option which uses a tool (CFPB Financial Well Being Scale) to define financial stability, using assumptions drawn from the general population, but not specifically from low income individuals and families. Financial well-being has different meaning for the low-income population in comparison to the general population.

f. Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US counties plays in addressing poverty. Increasing agency capacity is a very critical factor for achieving success in our programs and services.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

We have grave concerns regarding the burden of the proposed collection of information in light of a number of factors:

a. The current Information Services survey includes approximately 300+ reporting data elements. The proposed CSBG Annual Report includes more than 1,000 data elements. This significantly increases the reporting burden without providing new resources to assist the agencies in meeting this requirement.

b. The burden on local agencies is not impacted by the implementation of the On-Line Data Collection System. To comply with this would require significant modifications to our existing system.

c. Our management information system currently has the data elements required for CSBG IS reporting and some additional elements we as an agency have chosen to collect. To comply with ROMA NG Annual Report requirements would require significant (and costly) modifications to our existing system. In addition, our agency uses multiple and (and sometimes incompatible) funder required reporting systems which adds significant time to collect and report the data.

CRT’s Estimated Time Burden

The agency’s’ estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under estimated. Based on our calculations, the burden to our local agency would be 72,086 hours. The cost of this time burden is estimated to be $2,772,282.

CRT is a large community action agency – with a workforce of 700 employees, annual budget of $60 million; annual client service of 44,256 families and 112,206 individuals and around 60 different programs. Because of funder mandates and complexity of program services, the agency utilizes multiple databases to comply with program regulations and annual reporting requirements. Integration of these myriad of systems makes annual reporting more timely and costly. Increased reporting requirements and follow-up will impact our program and staff resources greatly.

3. The quality, utility, and clarity of the information to be collected
As noted below and in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

a. Several elements to be collected are not readily available and are the result of community wide efforts and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

b. Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it has little utility at an agency/local level; and it lends itself to meaningless data at a state and national level. It should be removed. The large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, our agency uses multiple reporting systems and it will have the added burden of tracking new customers across all programs and services.

c. Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

a. While this online automated system may benefit State Lead Agencies it will not be likely to benefit CRT as a sub grantee. We will still need systems that track customers, services, and outcomes. Given that many of our funding sources requires additional and sometime separate data collection systems and reporting systems that also increases the complexity of data collection.

b. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 45 (Del Norte Senior Center, Inc.; Eligible Entity/Local Agency in CA)

I am writing to respectfully submit my comments regarding the proposed Community Services

Block Grant (CSBG) reporting requirements familiarly known as “ROMA Next Generation.”

While I am specifically providing feedback on behalf of my own agency, I feel that my concerns are relevant to any CSBG-funded agency, especially one as small as mine.

Initially, I would like to comment that the development of the proposed reporting requirements was done very much without the ongoing input or participation of the individual agencies that make up the Community Action Network. Unlike the development of the organizational standards, which were created in a very open, inclusive and responsive process, the ROMA Next Generation product is the result of a closed and secretive process that has been completely unresponsive to the legitimate concerns of the CSBG network. I do not feel the agencies who are going to be bound by these cumbersome and unworkable requirements were given any meaningful opportunity to influence the outcome.

Overall, the ROMA Next Generation requirements have all the earmarks of having been developed primarily by theoreticians and researchers. The new indicators, demographic data collection and follow-up requirements appear to be a wish list of everything that it would be nice to know about individuals in poverty and the services they access. No thought appears to have been given as to whether there is a realistic possibility that the data could actually be collected.

Even rigorous scientific research only uses sample data. We as CSBG entities are being asked to collect research-quality data on every single person who is touched by CSBG funds. It is simply outside of the mission of the program and beyond the capacity of most agencies to achieve.

Thank you for this opportunity to comment, though it feels much too late in the process. My specific comments on elements of the proposed requirements are as follows:

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

The level of detail being required in both the community level initiatives and the family/individual NPI characteristics, and the amount of information required to be collected from beneficiaries is of limited use in the day-to-day functioning of the agency. It is not necessary for proper performance and has little to no practical utility.

Programs for which a significant level of detail is intrinsically necessary to provide the service will already have the collection of that data built into them. Programs that do not need extensive client demographic or follow-up information will be unduly hindered by excess data collection and reporting requirements. CLIENTS WILL AVOID SERVICES

AND AGENCIES WILL AVOID PARTNERING WHEN FACED WITH THE HIGH

AMOUNT OF DATA COLLECTION REQUIRED.

The existing NPI’s and client characteristics reports allow for sufficient data collection to inform the agency’s planning and performance evaluation activities and to provide program effectiveness data for federal purposes. There is no practical utility to having to report the level of detail being contemplated to a state or federal agency.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

The agency’s cost of the reporting burden of the proposed system is significantly underestimated, even for my small agency with no current sub recipients. As has been made abundantly clear in the comments that have been made to date, there is no single, unifying data collection system that can accurately and efficiently record and report the data currently being required, much less the much more detailed demographic, follow-up and narrative information being proposed. Our agency currently tracks CSBG data by hand, using client sign-in sheets and manual calculations. In an effort to reduce the reporting burden, we have sought out proposals for client tracking databases. Our agency is relatively simple, yet we still use three separate client tracking databases that report information across four distinct programs.

The estimated cost to purchase and set up a database system that would allow collection of unduplicated client information and integrate with the systems already in place was between $85,000 and $100,000 for my agency alone. Spending that much on a data collection system is completely unreasonable, given that this amount could fund half of my entire Senior

Nutrition program for a full year. If CSBG funds were used to fund sub recipients, the cost would be similar for each agency added to the system.
Absent a single unifying database, the burden to report data to every program in which a client could participate, then re-report it to CSBG multiplies. The table below illustrates our estimated labor burden to process our existing service population through all of the programs in which they participate, and to re-enter that information for CSBG’s proposed reporting requirements.

Total Hours Hourly Staff Rate Cost

Initial Intake 1,221 21.56 $ 26,324.76 $

Service Contacts 10,188 12.55 $ 127,853.88 $

Duplicate Entry 900 12.55 $ 11,295.00 $

Program Reporting 200 28.33 $ 5,666.00 $

Total Hours 12,509 171,139.64 $

TOTAL DATA COLLECTION AND REPORTING BURDEN

Assumes 30 minutes intake for each client annually and approximately 5 minutes of initial data entry per client service event. *Duplicate Entry estimates the hours required to enter data into a new CSBG data collection system that hasalready been entered into the data collection systems of other programs.

3. The quality, utility, and clarity of the information to be collected; and

The quality and clarity of the information to be collected is questionable. Most clients and staff persons are likely to accurately view the data collection process as simply a bureaucratic barrier to receiving and providing services. The utility of collecting and reporting individually identifiable data for every client for every service is also questionable. As was stated in earlier comments, even pure research activities use sample data. Requiring research-quality data on every client is unreasonable when our primary goal is not research, but service delivery.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

The people most in need of services to alleviate poverty are also the least likely to be able to access and use automated data collection. In addition, in an atmosphere where many programs are trying to minimize the barriers to service access, the collection of personally identifiable information is seen by many clients as being intrusive, especially among the hard
to reach homeless and those with mental health challenges. Asking questions about the sources of family income, race, employment status, household composition, etc. will often make people turn away from assistance.

Comment 46 (Delaware Opportunities Inc.; Eligible Entity/Local Agency in DE)

I. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

Module 3: Remove the Community Outcome Indicators that include percent or rates. Percent and rate of what? Without this information, (i.e. what is the base...where is it defined...when or where is it consistent) the result is meaningless. It will not mean much to us at the local level and when the data is aggregated at the state or national level it will mean absolutely nothing.

Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at this agency. We lack the data collection and reporting software to maintain the report and the cost for doing so is expected to be substantial initially and ongoing. We do not feel that the collection and reporting of this data will have any impact on our performance in the short or long term. We have not yet even grasped how to define one client as "new" and another as continuing. Our data system has households going back to at least 1990. If we served them as a Head Start child then and they are now applying for HEAP are they "new" now or were they "new" then?

2. The accuracy of the agency's estimate of the burden of the proposed collection of information

We believe that the estimate of the burden is substantially understated. First, it does not recognize that our existing data collection and reporting system will need to be vastly expanded or newly created. We anticipate that for the first year we will require an expenditure of approximately $10,000 in CSBG funds
just for the data collection and reporting tool plus a first year maintenance support cost of $6,000 or $16,000. We expect that there will be an ongoing cost of at least $6,000 each year for the maintenance of the new tool. The reporting tool referenced for use by the state will have limited and probably negligible use by an agency such as ours even if such a tool is eventually made available. All of the fields in such a tool would have to be uploaded from our local data base which would have to have the same fields. Hence the collection of data, organization, and reporting functions would need to be finished at the local level prior to uploading to a (probably superlative) reporting tool. Beyond this, we see a data collection, data entry, and data reporting requirement at 3 minutes per individual served. For this organization, this translates to 3 minutes times 10,000 unduplicated persons = 30,000 minutes or 500 hours. This does not include service log entries, and staff training. Ata minimum, this translates to an extra burden of $15, 188 (Average hourly wage of data collection/reporting staff @$22.50 plus fringe benefits at 35% = $30.375 x 500 hours), plus the ongoing $6,000 indicated above. For the first year this translates to a cost of $31,188 or 13% of our CSBG annual budget and Or about 9% charge of our current CSBG budget (more if there is sequestration or other budgetary reductions). The items to be tracked have increased by 330%. It has to be recognized that there is an additional cost associated with this and it is unlikely that anyone wants to increase budgets by upwards of 10% solely for administrative purposes. The result will be a reduction in services that the data collection is trying so hard to report upon.

3. The quality, utility, and clarity of the information to be collected; and

Module 4, Section A: Characteristics for NEW Individuals and Households — The data entry form, we believe is unnecessary, will produce no useful information and will be costly to implement. As noted earlier, the definition of "new" is subject to local interpretation and given the ambiguity of the term can
produce no information at the state or national level. Because demographic data alone does not include information about the needs of new customers, this information fails to help us determine if we are addressing issues identified in our community assessments. Furthermore, the large number of "one time" customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, we already use multiple reporting systems and will have the added burden of tracking new customers across all programs and services.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

The OCS comments which accompany the reporting tool indicate that the reporting burden on states will be lessened with the availability of an online reporting mechanism. That system, if allowed by New York State for use by us, will have little to no utility or purpose unless we develop a system which actually collects, aggregates, and summarizes in fields suitable for uploading to the online system. All of the burden for this will remain with us, the local agency. The auto populating feature OCS describes may be useful for State reports, but the ability to auto populate data on an agency by agency basis is probably not present. Since local agencies report to the State, the online system would have to be the State's system for data collection, or is it being proposed that this online reporting feature would be present for local agencies to use and obviate the need for the state collection and aggregation of data?

Comment 47 (Department Of Community Service And Development ; State in CA)

The State of California Department of Community Services and Development appreciates the opportunity to provide comment to the Community Services Block Grant (CSBG) Annual Report - Document Citation number 81 FR 39267. This effort is critical when considering that as a network we need to be able to effectively demonstrate outcomes and evaluate the programs and services that we administer. CSBG funding is an integral part of communities throughout the country, and through this funding an
array of supportive services are provided that help to lift low-income individuals and families out of poverty.

The Department acknowledges the substantial amount of work and effort that was put into forming this new performance management framework. However, it is important to note that new reporting efforts should include measures and reporting elements that provide additional benefit to CSBG program administration and delivery. With that in mind, there are areas that should be addressed prior to the implementation of this effort.

In providing this assessment, the Department not only considered its own interests, but also the interests of the California CSBG network. Our network is unique in that it is quite large and our local agencies strive to leverage CSBG to the highest degree
possible. This is a benefit to service delivery, but also poses some challenges when considering large scale programmatic and reporting changes. The California CSBG agencies have many funding sources, often times upwards of twenty or more funding
streams. Attached to those individual funding streams are specific requirements related to many different reporting systems. This fact would make it difficult for a local agency to move towards case management when several programs have different requirements.

The acquisition of a centralized data system for agencies would require a significant contribution in terms of staff time and agency resources to document the Information Technology requirements for all of their systems and then proceed to develop a single system to report this data up. Further, the agency would be responsible for deriving funds required to pay for merging the systems as well as developing a competitive solicitation to procure the single reporting system. When considering the provided timeline, there may not be sufficient time for agencies to cycle through the various steps needed to implement a new reporting system. There has been some discussion of how this would be achieved, but not enough discussion of how this effort will be funded.

Most importantly, this challenge does not take into consideration the amount of time and resources needed for state administering agencies to implement a centralized reporting system. While this may improve program integrity as a whole, there should be federal support through funding and training and technical assistance, as well as a more reasonable timeline to refine and implement reporting changes.

The Department is in support of increased efforts to evaluate the CSBG program and its local agencies. We encourage the ROMA Next Gen work group to continue their efforts.

Also, it may be helpful to leverage other human and social services outcome tools at the federal level, specifically programs such as Promoting Safe and Stable Families and/or Community Based Child Abuse Prevention grants. The hope is that the feedback provided will enhance the conversation around ROMA Next Gen.

Thank you for your dedication and commitment to providing support to those who serve low-income individuals and families throughout the nation. I look forward to continue working in partnership with you to strengthen our capacity to improve the lives of those living in poverty, and ultimately reduce poverty in the nation.

Comment 48 (Department Of Housing And Community Development ; State in MA)

The estimated Average Burden Hours per Response to the CSBG annual report for states (164 hours) is inaccurate and grossly underestimated. While we support the revised/automated forms and agree they will yield efficiencies in the State’s process for completing the annual report, which replaces the existing Information System (IS) Survey, the new process also increases State accountability. As a result, we expect to provide significant training to our staff and CAA network to ensure all understand how the different reporting pieces fit together and how to accurately complete the various report forms. We expect, due to the complexity of the report, training will be needed on an ongoing basis as CAA staff turnover and not just one time at implementation.

Additionally, the estimated of average burden hours of 164 do not account for time our State office will spend working with our vendor to completely redesign our data collection system so our CAAs can submit data needed for us to complete the annual report. We worked with our vendor last year to design and implement a new online system to address another aspect of OCS’s Performance Management Framework, Organizational Standards, and the project took nearly 5 months.

•Identify financial resources and/or other means of supporting States and their CAAs in modifying data systems and building “data bridges” with other funding sources, particularly for Federal programs (e.g. – Head Start, WIC, HUD, SNAP, WIOA, etc.) The CSBG Annual Report accounts for a complex network of services and strategies but what it ultimately strives to answer is: what mix of services/strategies are needed to move people towards income security and out of poverty? We can’t answer that question until our network has a holistic view of data from (at least) all the major federal funding streams.

Comment 50 (Department Of Social Services Office Of Community Services ; State in CT)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

3. The quality, utility, and clarity of the information to be collected; and

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Comment 51 (Dickinson Community Action Partnership ; Other/Unknown in ND)

Community Action Partnership of Dickinson, North Dakota (CAP) is respectfully submitting the following general comments regarding the CSBG Annual Report OMB Clearance Process. The overall opinion of our Agency regarding ROMA NG is that it is overly and unnecessarily burdensome, expensive, and does very little to enhance the experience of clients and their efforts to obtain self-sufficiency

While there are elements that may be helpful, there are a number of areas detailed below where the general collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

Address the unique issues of Public entities that receive CSBG. Public agencies (sub-grantees) that sub-contract most or all of their CSBG funding to other service providers may face added challenges of requiring their grantees to collect data in accordance with ROMA Next Generation requirements since it may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet general, agency goal the reporting requirements. This challenge is not addressed in the proposed reporting package.

Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US
counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network for ongoing training, technical assistance, and general capacity-building supports.

2. The accuracy of the agency's estimate of the burden of the proposed collection of information

Regarding the accuracy of the agency estimate of the cost burden, CAP concerns include: general
The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

The burden on local agencies is impacted by the implementation of the Online Data

Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems. An initial investment in new software for North Dakota was quoted at over $50,000 which only covers the set-up and one year of training. This does not include any of the costs associated with adding necessary modules such as Weatherization or Head Start.

Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data.
In addition, it is not clear how the burden for local agencies is decreased by the online data collection system described in the by OCS. OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and — at the discretion of the states — the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)." It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden outlined here.

3.The quality, utility, and clarity of the information to be collected

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

• As noted above, OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and — at the discretion of the states — the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)." It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and "talk" to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 52 (Dickinson Iron Community Service Agency ; Eligible Entity/Local Agency in MI)

We at DICSA, a community action agency serving the rural counties of Dickinson and Iron in Michigan’s Upper Peninsula, are concerned that the ROMA NG package is overly burdensome and not attainable within the scope of our Community Services Block Grant. For instance, the goals of this report don’t seem to consider smaller agencies with no urban center and the minimum level of funding. Our staff is stretched to its capacity and cannot possibly take on full scale case management or rework our community partnerships to fit into the specific strategies of Collective Impact. Instead of helping our agency performance, the report serves instead to draw essential staff away from impacting low income individuals’ self-sufficiency in order to adapt what we do to fit the rubrics and metrics of another set of reporting guidelines.

The estimated burden of the proposed collection of information is incredibly challenging to quantify. How much will this report change what our intake staff collect and input into our data systems? Will these systems have the capacity to gather the information and to effectively sort and report the data? How much time will be spent on learning the new reporting data elements, connecting them with the current elements, and putting them into quantifiable and meaningful terms? Will any of this help us do a better job serving our local residents? The estimated time burden of 242 hours per Community Action Agency is likely under estimated. Those 242 hours are probably what will be needed for one person to fully understand the new system, train others on what is needed to be gathered, and eventually be able to collect, calculate, decipher, formulate and report the data. The cost of this time burden to our agency for that one person would be $8,983. We fully expect that numerous additional hours will be expended by intake staff and program managers, adding to the burden. This increased burden of reporting comes with no new resources. Smaller agencies like ours simply do not have the cash flow or general fund to support non-budgeted expenses.

Comment 54 (East Missouri Action Agency, Inc.; Eligible Entity/Local Agency in MO)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

I agree that the collection of data is necessary to substantiate the work that we do, and prove that our work is valid. I also feel that the community action network in Missouri is ready to begin the process of ROMA Next Gen, but I am worried that many states are not up to the task yet, which can potentially be very harmful to community action, overall. If the other states do not have the resources needed to collect the required information, how will that affect the entire network when legislators and other officials review the work we have done?

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information.

Considering that the current IS report includes 300+ data elements to report, and the proposed CSBG Annual Report includes more than 1,000 data elements, I believe that OCS’s estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is somewhat under estimated. Based on our calculations, the burden to our agency would be 396 hours. The cost of this time is estimated to be $8,740. Please note, this estimation does not include the T/TA that will be needed for our employees to understand how to complete the new report.

3. The quality, utility, and clarity of the information to be collected.

If the proposed number of general indicators were decreased, for example, 2-4 general indicators for each of the domains, then the aggregate data collected from the entire network would be more focused and refined. Which would then be much more meaningful data to present to legislators and funders? To show more localized impact, agencies should still be allowed to track other outcomes using local indicators more specific to their unique programs and communities.

For more clarity in the information collected, I feel that the data would have more impact if the indicators were worded similarly to other programs, such as Head Start and HUD. If all programs reported using similar indicators, then, community action would have more impact on a national level.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

In Missouri, we have been using a state-wide data system for many years. Several other states do not have this resource, which is absolutely necessary for collecting accurate data. The proposed on-line automated system is a nice thought, but much of our agency’s reporting comes from many different data tracking systems. Our other programs, Head Start, Women’s Wellness, Housing/HUD, and Weatherization all have their own systems. Unless OCS can develop a system that can be used in conjunction with all of the other systems, the proposed on-line automated system will not work for local agency. Therefore, the burden of the collection of information at the local level will not be decreased in any way. It is still going to be a large and expensive burden on our agency.

Comment 55 (Florida Department Of Economic Opportunity ; Other/Unknown in FL)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

3. The quality, utility, and clarity of the information to be collected; and

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

* It is this agency’s opinion that overall, the proposed ROMA Next Generation changes are positive and a move in the right direction. However, it is also my belief that some of the additional indicators will be difficult to track as it can be almost impossible to measure some of the conditions so closely due to external situations that are beyond our agency’s control.

* Implementation Timelines – being that we are a public agency, our turnaround time on such a project will very likely be slower than private agencies. All aspects of our department are required to be reviewed and scrutinized by other areas such as budget & management, procurement, etc. We may not always be able to meet our deadlines due to situations that are beyond our control.

* Staff Training Needs – it goes without saying that these changes are detailed and labor intensive program. Additional training will be required.

Comment 57 (Fresno Economic Opportunities Commission ; Other/Unknown in CA)

We are writing to respectfully request that the proposed ROMA Next Generation (ROMA NG) be deferred for introduction to the new administration, its review and comment, with a concurrent, much more thorough network assessment, comment, and road test. We have attended all available webinars hosted by OCS and NASCSP and, to my knowledge, are not aware – until this time – that there has been a formal, public demonstration of the software ROMA NG intends for us to use for the collection of data. That alone causes considerable concern; we are not prepared to provide complete and informed comments without having experienced the data collection systems ROMA NG proposes.

While we offer a succinct summary of our assessment and deep concerns over the proposed changes to ROMA below, it seems clear that the underlying agenda behind the introduction of ROMA NG is contrary to the authorizing language of the Economic Opportunity Act of 1964 and successive re-authorizations of that Act.

The Act emphasizes the local focus of Community Action, something virtually foreign to existing government trends that centralize and control more and more of the activities of our communities. The Community Action Network is successful precisely because each jurisdiction is given the mandate and freedom to assess local needs, and direct its resources to needs prioritized by the input of local residents, leaders appointed by elected officials representing the jurisdiction, and local constituents elected by local residents. As government adds to the layers of administration and bureaucracy entailed in community action and the expenditure of CSBG funds to carry out the work of Community Action, a perilous, slippery slope towards stripping agencies of the local control that is at the core of our success will occur. Each community, each region in our nation is unique. It is the hallmark of Community Action to empower local leaders and consult local residents to design and develop customized programs and initiatives to fight poverty.

Furthermore, the timing of the introduction of ROMA NG and the short timeframes to comment on an initiative that adds significant components and layers to the existing system make it appear that the administration is eager to install the new system before it leaves office. Our concerns over the extraordinarily burdensome and unnecessary new requirements of the system require that the intent to usher in ROMA NG be deferred until the new administration, working hand-in-hand with the network, can reassess the need for a revised ROMA system. The system in its current configuration satisfies requests for information made by our elected leaders, funders, and the residents of our communities.

While we understand and agree entirely with the need for accountability, updated by the administration in 2011 in the form of the Government Performance and Results Act (GPRA), the proposed ROMA NG requirements are simply overbearing, unnecessary, and introduce inordinate time and expense commitments to the detriment of the energy and focus that our mandate to create opportunity and reduce poverty requires.

The data to be collected by ROMA NG is significantly more than the existing ROMA requirements which, for an agency as complex as Fresno EOC, would be extraordinarily burdensome, expensive, and time consuming. The same concerns would apply for smaller agencies that would not have the capacity, staffing, infrastructure, funding, or time to complete the required reports. A sampling of added elements in ROMA NG give rise to our concern.

d. After extensive research and retaining qualified consultants to advise us in the use of electronic systems to collect, collate, and report unduplicated data across our 14 programs, 49 funding sources, 101 contracts, and 42 funding periods, it became apparent the cost to purchase and maintain such a system would be prohibitive. Estimates were in excess of $270,000 per year, all of which would have to be absorbed by CSBG and would reduce program activity to report activity only CSBG requires. Our board elected not to move forward with the plan. Any reliable collection methodology to meet the increased burden of ROMA NG reporting, whether manual or electronic, will add significant cost, funds we would much rather deploy for direct services and/or leverage for greater impact.

2. Those who are familiar with the kinds of information, and volume of information Congress seeks from the network report that much of the data ROMA NG would collect is superfluous. Congressional requests tend to be more general, such as, “How many families did you move out of poverty?”

3. Will implementing ROMA NG help agencies do their job better? The work of community action is fluid and extremely challenging. It requires that we be resourceful, nimble, and innovative. Is it data that generally informs new directions or new responses to urgent needs? Not always. Perhaps a better barometer of need is information drawn directly from residents living in our disadvantaged communities. The involvement of our residents in the work of community action is a core value of community action; in our experience, low-income residents provide us with the best guidance on need, priorities, and resources they require.

Recommendations

We would suggest a program and process-based approach to document activities and outcomes, a much simpler format aligned with the way in which data is more commonly collected by agencies. With this approach agencies would provide data and anecdotal highlights for major programs they operate. Data would be drawn from reports the agency already submits to funders, stakeholders, and constituents.

Reports would be unique to each community action agency which, after all, is the hallmark of community action. We are among the few remaining federal programs that enjoy bipartisan support precisely because, at our core, Community Action is about local control, it requires and honors the involvement and wishes of local leaders – including representatives of low-income communities – rather than foisting a predetermined agenda and menu of services to report on. To merge the thinking and actions of agencies toward a raft of established activities and outcomes runs the risk of derailing efforts to respond to local need in innovative ways.

The system of measurement and accountability should drive agencies with the urgency to think broadly, ever seeking innovative solutions that meet local needs. Responding to a prescribed menu of activities and expected outcomes limits innovation, it threatens to suffocate innovation and the creative advances that demonstrate new hope and opportunity for those living in poverty. In other words, the more defined, the more extensive and prescriptive the measuring tool, the less likely innovation will occur.

Now is the time to innovate, now is the time to open the vast reservoirs of thought and ideas to demonstrate the value and worth of community action. Now is the time to herald our core values – the uniqueness of each community and the leveraging of CSBG dollars to innovate new

Comment 58 (Greater Bergen Community Action; Eligible Entity/Local Agency in NJ)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

On an individual program level, most of the information is already collected and analyzed to ensure that the program is necessary, efficient, and changing the lives of the individuals enrolled. The many funders of CAA programs already require outcome reporting, but, of course, in their own format. Compiling all the information for this annual report is a goal to reach for, but currently not practical. Most state and federal funders of programs run by CAAs do not share client data files with anyone – gathering information from each program is time consuming when possible. The cost-effective technology needed is simply not available to individual agencies.

Module 3 I think falls short of the goal of being rolled-up to better present a nation-wide picture of community action. If we each define our community, the rates and percentages have no meaning when combined on a national level – how can a 10% increase on a block level be aggregated on a state level? The tools indicated are for project management, and should be a tool for agency use, not a required report.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

Without a definition of what specifically is included in the estimate of the burden, and the assumption that it should include the preparation required throughout the fiscal year to allow for one employee to collate the information required for the report, the estimate falls very short of the time needed. Our current costs to complete the IS Survey, gather all the information throughout the year and maintain the technology required is around $6800. That covers 222 employee hours. Our estimate for the burden for the proposed report is $24,200, including 658 hours. The hours will almost triple due to the increase in the amount of information required to be tracked for community projects, collected and analyzed for all client outcomes, training on new technology needed for compliance, time needed to complete the more complex reporting forms, etc. The database technology this makes necessary increases that cost substantially.

3. The quality, utility, and clarity of the information to be collected; and

The Individual and Family level NPIs have been well-though out and improved. The community level reporting has issues with definitions, rates and percentages that need to be more fully explained. The report on services delivered is a list of things we do, not outcomes. Although useful for an agency to know, it doesn’t advance our national story, and should not be in the report. The same thing applies to the list of community strategies.

Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

The interactive spread sheets will minimize the burden after the first year. Getting all the federal and state databases to allow agencies to download and/or upload information into another database would reduce the data collection burden tremendously.

Comment 59 (Greater Bergen Community Action ; Eligible Entity/Local Agency in NJ)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

On an individual program level, most of the information is already collected and analyzed to ensure that the program is necessary, efficient, and changing the lives of the individuals enrolled. The many funders of CAA programs already require outcome reporting, but, of course, in their own format. Compiling all the information for this annual report is a goal to reach for, but currently not practical. Most state and federal funders of programs run by CAAs do not share client data files with anyone – gathering information from each program is time consuming when possible. The cost-effective technology needed is simply not available to individual agencies.

Module 3 I think falls short of the goal of being rolled-up to better present a nation-wide picture of community action. If we each define our community, the rates and percentages have no meaning when combined on a national level – how can a 10% increase on a block level be aggregated on a state level? The tools indicated are for project management, and should be a tool for agency use, not a required report.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

Without a definition of what specifically is included in the estimate of the burden, and the assumption that it should include the preparation required throughout the fiscal year to allow for one employee to collate the information required for the report, the estimate falls very short of the time needed. Our current costs to complete the IS Survey, gather all the information throughout the year and maintain the technology required is around $6800. That covers 222 employee hours. Our estimate for the burden for the proposed report is $24,200, including 658 hours. The hours will almost triple due to the increase in the amount of information required to be tracked for community projects, collected and analyzed for all client outcomes, training on new technology needed for compliance, time needed to complete the more complex reporting forms, etc. The database technology this makes necessary increases that cost substantially.

3. The quality, utility, and clarity of the information to be collected; and

The Individual and Family level NPIs have been well-though out and improved. The community level reporting has issues with definitions, rates and percentages that need to be more fully explained. The report on services delivered is a list of things we do, not outcomes. Although useful for an agency to know, it doesn’t advance our national story, and should not be in the report. The same thing applies to the list of community strategies.

Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

The interactive spread sheets will minimize the burden after the first year. Getting all the federal and state databases to allow agencies to download and/or upload information into another database would reduce the data collection burden tremendously.

Comment 60 (Hinds County Human Resources Agency ; Other/Unknown in MS)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The information has practical value for the local agencies but offers no practical value to combine at the state or federal level. The data collected now has not be used for any other purpose than reporting. The increased information assumes the purpose of Community Action Agencies to be uniform or follow the vision of one individual. This counters the original purpose of Community Action to be locally driven to fits the needs of each community. While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

Furthermore, the introduction of “Collective Impact” has been annoying a lot of marginalized communities because it is yet another example of the mainstream community “discovering” something that has been around for a long time, slapping some academic labels and concepts on it, positioning it as new, and then getting all the attention and resources. Long before CI came out as a formal concept, community action in diverse communities were already working together, sharing information, creating coordinating mechanisms. We have been doing Collective Impact way before it was called that, and we continue to.

This new community format also perpetuates Trickle-Down Community Engagement: this is where “we bypass the people who are most affected by issues, engage and fund larger organizations to tackle these issues, and hope that miraculously the people most affected will help out in the effort, usually for free.” Without meaning to, CI backbone organizations have become some of the biggest perpetrators of this terrible and destructive practice. Some backbone organizations’ gravity is so strong that it begins gatekeeping, spewing out tiny amounts back to the organizations and communities most affected by the challenges the CI efforts are trying to address. Grassroots organizations that used to be able to solicit significant participation directly from people of low income are now told from whatever juggernaut CI effort is ruling the landscape.

Due to all the gatekeeping, inequitable funding allocation process, TDCE, and other factors, communities of color are oftentimes left behind by CI efforts. We are not funded on the same level, and yet are asked to provide input, do outreach, mobilize our communities, etc., and because we believe in the goals of the CI efforts, and sometimes because we are backed into corners, we’ll be involved. This leads to the CI effort being seen as inclusive, when in reality, it is tokenizing. This creates a shell of our purpose, our need and our ability to give voice to those we serve.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

The agency’s’ estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under estimated. Based on our calculations, the burden to our local agency would be 520 hours. The cost of this time burden is estimated to be $24,000. The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources. The burden on local agencies is not impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing data collection systems.

This proposed data information especially in the community level section uses an incorrect perception of the Community Action Network purpose and therefor drives the agencies burden in a completely different direction then developed over the lifetime of the agency.

3. The quality, utility, and clarity of the information to be collected; and

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level. The continue vision of Community Action Agencies as some sort of research institution does not allow for the rigorous amount of stabilization and self-sufficiency case management or community development work that is needed for outcomes for everyday low income individuals. The quality of data listed in this new system creates a fantasy albeit also dangerous collection process. Experimenting on impoverished people was never the intent of CSBG. It is develop real solutions with local induvial development.

Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading and extremely dangerous data collection.

This will hold agencies accountable for such factors in which they have little control over.

Take a state such as Mississippi that continues to reject multiple systems and programs that would greatly impact those in which we serve. This state rejected Medicaid expansion while being the unhealthiest state. This state reduced education funding while being rank 50th in multiple education factors. This state continues to limit all forms of birth control and birth control education while leading the leading persons in poverty include young single mothers. Therefor the work of our local agencies become that of much survival and small but mighty impact on the individual level while also allowing many of communities to not just fully collapse under the pressure of rising cost with low wages. Crime rates rise as a means of poverty survival and even serving as the biggest CSBG recipient in the state, we have to focus on the individuals we can change and not always research of collective impacts.

· Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry

Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

· Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

As noted above, OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 61 (Illinois Association Community Action Agency ; Other/Unknown in IL)

This letter provides detailed comments and recommendations based on the format requested in the Federal Register. Overarching concerns with the currently proposed CSBG Annual Report include:

Reporting requirements that exceed agency capacity for data collection, integration, and analysis.

Many agencies struggle under the burden of collecting and reporting data for multiple programs that use and require different — and often incompatible — software and systems. The CSBG Annual Report as currently proposed is unduly burdensome and underestimates the time required at the local level to collect, integrate, and report data at the agency level.

Reporting requirements that produce data of limited utility for CAAs and State CSBG Lead Agencies. IACAA strongly believes that all data collected for the CSBG Annual Report should have a clear purpose and use for local agencies and State CSBG Lead Agencies. Caution must be taken if data reporting is included that is not intended to be aggregated and used at the state and federal level.

Responses to the Questions

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The Illinois Association of Community Action Agencies (IACAA) agrees with the Community Action Partnership that while there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide unnecessary information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility.

Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

I. Module I: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported and creates a misuse of the Organizational Standards.

Expecting constant perfection is unrealistic and does not have practical utility for local agencies or the State CSBG Lead Agency. As proposed, the single metric the State Agencies will report on will be the number of eligible entities that meet 100% of the CSBG Organizational Standards. To increase utility of the information, State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate
Illinois Association of Community Action Agencies Feedback — CSBG Annual Report OMB PRA Clearance FRN #1
strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance State Agencies should be able to report the number of local agencies that have met 100%, 9() %, 80%, 70%, 60%, 50%, less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency. This is not necessary for agency performance, and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will be skewed by large population centers and decrease the practical utility of the data even further. Furthermore, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems, In addition, as many local agencies serve families over time, with no definition of "new" being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years (e.g. Early Head Start, child care, after school programs, youth programs, job training, WIC, food boxes, LIHEAP, etc.).
Module 3: Remove the Community Outcome Indicators that include percent or rates. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach is inappropriate and unnecessary given the broader range of options for doing community-level work. Collective Impact should be removed and included in training and technical assistance endeavors as a potential local option. Besides CAAs would need significant resources and support to engage in Collective Impact which is presently not the case.

Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the eligible entity's work. One option duplicates data already collected and the second uses a tool that is not specifically designed for low-income individuals and families and has an overly broad survey to define Community Action customer stability.

Address the unique issues of Public entities that receive CSBG. Public agencies that sub-contract most or all of their CSBG funding to other service providers may face added 3Illinois Association of Community Action Agencies Feedback — CSBG Annual Report OMB PRA Clearance FRN #1 challenges of requiring their grantees to collect data in accordance with ROMA Next Generation requirements since it may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet the reporting requirements. This challenge is not addressed in the proposed reporting package. Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results.

Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. Adding back an agency goal is a prudent approach. The standards will not capture or encourage all of the work of CAAs. The best example is the OCS directive to use bundled services and to examine/address the impact of toxic stress on young brains. When addressing toxic stress, CAAs with Head Start programs will be better equipped to explore and offer some services; the bottom line is that most CAAs do not currently hold the capacity to address the impact of toxic stress. Collaborations with early childhood organizations are essential to move in this direction, as well as partnerships with universities and other educational institutions. The argument can be made that the Standards provide some direction, but does not go far enough to support this work. This will require a goal for agencies to build capacity and expertise to integrate new practices that addresses poverty. This requires systemic change that goes beyond case management. CAAs are exploring bundling methods that can provide better services, but require unduplicated counts to multiple funders. This can be challenging and requires that agencies enhance or change processes and have the ability to upgrade when technology advances.

This also highlights the broader need of the Network to commit resources for ongoing training, technical assistance, and general capacity-building supports consistent with private sector approaches such as Continuous Quality Improvement, Lean and Six Sigma.

2. The accuracy of the agency's estimate of the burden of the proposed collection of information

While this is an exciting area for Community Action, there needs to be an identification of common practices across the entire network. Collection of data will be a huge undertaking, but at this point, much of the data analysis will be beholden to the expertise, either at the agency or association level, which could be varied and not necessarily rigorous. Standardized processes, such as the Needs Assessment and a data collection tool, beyond the NPIs, would go a long way to allow this type of collection and analysis to occur. Additionally, CAAs may not have adequate resources or capacity to support the collection, analysis and use of data. At best, it is not clear how the burden for local agencies is decreased by the online data collection system described in the by OCS. Consequently, if the time burden analysis is to be conducted with the scenario of successful implementation, then it seems prudent to also perform a scenario analysis of time burden with an assumption of no new system. In the event of failed implementation, substantial additional state and federal level resources would be diverted away from capacity building to support high-quality programs toward data collection.

Illinois Association of Community Action Agencies Feedback — CSBG Annual Report OMB PRA Clearance FRN #1

Concerns include
The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

The burden on local agencies is not alleviated with the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems which will come at a premium cost with no additional resources.

Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data.

3. The quality, utility, and clarity of the information to be collected; and

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of "one time" customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide
Illinois Association of Community Action Agencies Feedback — CSBG Annual Report OMB PRA Clearance FRN #1number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

As noted above, OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and — at the discretion of the states — the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)." It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and "talk" to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 62 (Inca Community Service; Eligible Entity/Local Agency in OK)

Question 1: Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility
Response:

The proposed collection of system in extremely burdensome and expensive to the point we would be spending more time and money rendering reports than assisting those in need become self-sufficient. In reviewing the module we found some useful tools and reports but there were many which seem redundant and provided useless information for us to use in improving the agency and the quality of services we provide.

· Module 1: Change the reporting on Organizational Standards


Expecting constant perfection is impossible and unrealistic. The Organizational Standards are more of a living document which is always changing, with different standards becoming due at different times. It is unrealistic to believe that an agency could be constantly at 100%.

A more realistic, accurate and useful tool would be for the State Agencies to report the number of local agencies that have met 100%, 90%, 80%,70%, 60% 50% and less. This would give a clearer picture of where and how much technical assistance is needed.

· Module 3: Community Outcome Indicators
Again we looked at how this information who assist us and found the information distorted and ineffective. Our service area contains cities and towns under the population of 5,000 and 100’s of pocket communities of 100 or less. This data was meaningless and incomplete since are agencies target area was smaller populations and spread over many miles. Because of the size of our target population the data on rate change at the community level on social indicators could not accurately be displayed or useful. Some flexibility in choosing the indicators used, using geographical makeup and description of target service area and goals would give a more accurate picture.

· Module 4: Remove the NEW Individual and Household Staff

The other issue to the time and expense of collecting, sorting, and ensuring this data is correct. Currently we do not have an effective system or manpower to handle this type of input or ensure the data is correct. This is a lot of expense for data which can easily be distorted or of very minimal use.

Module 4: Remove Stability Indicators

What is stable in a rural community in Southeast Oklahoma is completely different from what is stable in New York City. There is also too much difference in the interpretation of what financial well-being means to put a one size fits all scale to.

Question 2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.

As an agency we are already feeling the burden and stress of reporting on the current system (374). It will be overwhelming to nearly triple the now-required information to 1131 items. This would be an overwhelming addition to administrative cost, data entry costs and IT costs which would greatly effective the funds needed to provide the services to those in need. In a time where we are trying to more efficient and invest our funds into the projects and families who need them, it does not make sense to spend so much time and effort in generic reporting which will not show each Community Action Agency accurately.

Below is the estimated time of 242 hours for local Community Action Agencies is significantly underestimated. Our agency is currently stressed fulfilling the current ROMA without the extra burden. Based on our calculations the burden to our local agency would be 1,836 hours before adding the estimated hours for follow up which would be an additional 1,875 hours = 3,711 hours. The cost of this burden is estimated to be on an average of $25.00 an hour (wage + benefits) = $45,900.00 with additional follow-up requirement $92,775.00.

Our Methodology Used to figure cost above what is currently being conducted:

· The amount of time of intake and other workers will spend obtaining information that is not required by other programs. 5 hours a week x 52 = 260

· Amount of time entering data or transferring it from another program’s software to the CSBG information system.

Entering Data - 15 minutes per entry (service) X 4,000 services = 600 hours.

Transferring Data – RSVP (1 hour a month) x 12 = 12 hours; Wk, RxOklahoma (1hour a month) = 12 hours; RAVE – (10 hours a month) x 12 = 120 hours; JAMM transit – (30 minutes a day) = 480 per year. Total of all Transferring time = 624 hours

· Typical Training Time spent for each worker to me trained and refreshed on the skills of data entry, or data acquisition, including new and continuing staff. Based on current training time and turn-over in positions. 24 hours

· Time spent team will spend on data quality assurance, as well as quality assurance on the final report. 4 hours for Quality Control personnel, 2 hours for report generation, 2 hours for corrections. Monthly team meeting review and discussing date: (1 hour X 12 = 12 hours). Final Review and preparation for Board of Director. ( 1 hour per month) = 21 X 12 = 252 hours.

· Time spent tracking individuals and families, tracking progress of community projects and recording the progress in the format required in Modules 3 and 4. Average Estimate time per client follow=up is 1.5 hours x 1,250 unduplicated service recipients (estimated # which currently do not have a follow-up process) = 1,875 hours per year. (Used historic data on a current case management program to estimate amount of time on follow-up on to come up with the average time). We do not currently do follow-up on a large portion of our service recipients due to the expense.

· Time spent gathering external demographic information required in module 4. Estimated 8 hours. (Based on time spent on gathering data for CAN)

· Time spent tracking volunteers, board members and their commitments to organization and project that are not run by your Community Action Agency. 1 hour per month for Board = 12 hours 2 hours per month for volunteers = 24 hours. A total of 36 hours.

· Time spent on additional analysis required to report of the 50 organizational standards. 2 hours per month = 24 hours

· Time spent reporting to state and answering the narratives required by the state for module 1, as well as the local information on funds, people and projects. 8 hours


Question 3: The quality, utility, and clarity of the information to be collected; and

· As mention above our agency’s service area covers four counties in rural Oklahoma which does not have any cities or town above 5000 and many small communities with populations of 200 or less. Some of the information to be collected is not available for our geographic areas and incomplete in other. This would result in even additional time being used to create the data necessary and ensuring it is accurate. This would especially be true in the social and community indicators needed.

· Our management team discussed in length how we would use this information as an agency and the only use we could determine was whether more individuals needed the service this year than from a previous year. This could be achieved with just a check box. All the detailed characteristic information seems impertinent to the needs of our community or the services we give. Who is new? Every time someone changes partners are they new? Are they new to the agency or the program they are participating in? We believe this will cause the final results to be faulty information.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Our agency currently uses tracking systems from 6 different funding sources (Head Start, RSVP, Oklahoma Department of Transportation, Homeless Program, and Oklahoma Department of Human Services) using their data input system. This also includes the data tracking system for our CSBG program called CAPTAIN. None of these systems “talk” to each other so we spend time doing double entry putting the information into the CSBG reporting system. Most of these are federal program, if a system could be put in place so these could communicate with each other this would greatly reduce the manpower time it takes to do double entry.

Comment 63 (Indiana Housing And Community Development Authority ; State in ID)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

3. The quality, utility, and clarity of the information to be collected; and

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Comment 64 (Iowa Community Action Association; Eligible Entity/Local Agency in IA)

While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet a threshold of usefulness. The reporting tools as proposed are overly burdensome and much of the data to be collected is outside the reach of the Community Services Block Grant (CSBG). Several agencies in Iowa receive a minimal amount of CSBG and yet are expected to accomplish all that is proposed in this package, compared to other agencies that receive a good portion of the funds that are expected to accomplish all that is being proposed (which the argument can still be made that the expectations being laid out to capture data across the entire agency and not just with the CSBG funded program is OCS stepping outside the scope of their authority). However, we do believe with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility.

1. Module I: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses.

We are not afraid of accountability and not opposed to the Organizational Standards, however we know it is hard to get from A to Z without a few steps in the middle, so having the option to report the number of agencies at a variety of thresholds is requested. This will allow for a more accurate reflection of agency performance. For instance State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the ICAA is the premier force ensuring those with less achieve more standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment in training and technical assistance.

Module 3: Community Level

We see the value of pushing for more community work because we know we cannot alleviate the causes and conditions of poverty alone. Given that belief, collecting percentages and rates towards success is not the way to measure the work local agencies are performing. Community Action Agencies are required to meet the needs at the local level and those needs vary so greatly that creating the kind of system that fits us in a box or holds us accountable for larger social issues is not appropriate.

In regards to tracking community level work as collective impact, partnership or independent types of community work, we do not see the need to be specific about the approach being used but rather allowing agencies the flexibility to report on outcomes being achieved.

Module 4: Remove the Characteristics for NEW Individuals and Households Report.

Our network has not been convinced how this information is useful to the agency, or network as a whole, in relation to measuring performance.

Financial well-being is only one piece and not appropriate as a standalone measure for stability. Meeting basic needs in the moment is often what helps stabilize a family and that could be through numerous means such as utility assistance or providing a family with nutritional food. The suggested stability indicators do not appropriately measure the number of families stabilized by community action.



Question 2:



The accuracy of the agency's estimate of the burden of the proposed collection of information.



As a State Association we do not have specific calculations on the estimated burden to collect the proposed data at the local level, but the overwhelming comments from our Iowa network has been the concern of increased expectations from OCS with no additional funding support for staff time or to ensure quality management information systems are in place. There is not an agency in Iowa that does not want the best IT system possible. But it takes resources we do not have. Requiring the collection of



this data without providing a quality data collection system or the resources for local agencies or states to create their own, is problematic. It is not an option in Iowa for the State CSBG Lead Agency to use discretionary dollars for such a system, as the Iowa Code mandates a four percent administrative rate limit and no discretionary funding. We want the best data — quality, accurate, consistent data across the state/nation — however there is not funding to support this need.



In addition, it is not clear how the burden for local agencies is decreased by the online data collection system described by OCS. OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line ICAA is the premier force ensuring those with less achieve more automated system for use by the states and — at the discretion of the states — the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)." However, on the July 14, 2016 webinar hosted by NASCSP, an OCS staff person said that there is 'no plan for a nationwide data system' and that OCS wanted to see 'what can states do using discretionary money'. It is important to note that even if an online automated system was created, it may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) that require additional and sometimes separate data collection systems — it is a real challenge to be able to collect unduplicated, accurate data across an agency. And as stated above, it is not an option in Iowa for the State CSBG Lead Agency to use discretionary dollars for such a system, as the Iowa Code mandates a four percent administrative rate limit and no discretionary funding.

Question 3: The quality, utility, and clarity of the information to be collected.

Urban areas may have greater success collecting some of the data elements that are being suggested but the reality in rural America is collecting this data would be extremely difficult in smaller geographic and more densely populated service areas.

Module 4, REMOVE Section A: Characteristics for NEW Individuals and Households - Data Entry Form For years, Community Action Agencies have adjusted, work to address the requirement of multiple reporting systems and this data entry form is a level of reporting that we do not see as necessary or helpful for telling our story of helping people, changing lives.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.
The Iowa network faces a burden that affects most of the 1,000+ CSBG Eligible Entities, in that OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. This makes it nearly impossible to track an unduplicated count of individuals served by each agency — as some systems, such as WIC, prohibits their client information be shared with a Community Action Agency's main database system — even when it is a program of that agency! In order to minimize the burden on respondents, at the least, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field use data collection systems that can be easily integrated and "talk" to one another to produce reports that meet the needs of local agencies, states, and OCS. Ideally an agency with any combination of federal, state and local programs would be able to maintain an overall agency database to track unduplicated individuals and families being served by each program.
As noted above, it is not clear how the burden for local agencies is decreased by the online data collection system described in the by OCS. OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and — at the discretion of the states — the ICAA is the premier force ensuring those with less achieve more.
Local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)." However, on the July 14, 2016 webinar hosted by NASCSP, an OCS staff person said that there was 'no plan for a nationwide data system' and that OCS wanted to see 'what can states do using discretionary money'. It is important to note that even if an online automated system was created, it may benefit State Lead
Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) that require additional and sometimes separate data collection systems — it is a real challenge to be able to collect unduplicated, accurate data across an agency. And as stated above, it is not an option in Iowa for the State CSBG Lead Agency to use discretionary dollars for such a system, as the Iowa Code mandates a four percent administrative rate limit and no discretionary dollars.

Comment 65 (Kceoc Community Action Partnership; Eligible Entity/Local Agency in KY)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

Collection of information is always necessary to measure performance and to ensure accountability. In the case of the changes, the feedback opens up whether or not the information collected is beneficial and necessary to really tell our story. The IS report is about outcomes, not process; therein lies our part of concerns with the report changes. There are positive changes; however, there are others that will create undue hardship on agencies already strapped for funding and maxed on staff capacity. We are already reporting on the ROMA report, logic model (reported internally, since required by the state to collect with the annual proposal), and the IS survey. Agency wide, we are inputting several thousand families in our agency wide database. Then, we have inputting for other programs leveraged/supported by CSBG and other funding. KCEOC has some staff already inputting information for the same clients into 4 separate databases. It is overwhelming to say the least.

As a CSBG Director and Director over all of these other programs, I am then charged with overseeing all this data, pulling reports, completing reports, ensuring accuracy, maintaining compliance, checking files to ensure the data from the files matches the data from databases; and, this is only a minor part of my day to day duties. I am also concerned with how accurate the reporting will be since staff will, for example in reference to Module 2 Section B, have to choose between properly serving the client and documenting time spent on administrative type duties such as planning. In my agency, we always try to put the client’s needs first. When we do this, documenting hours for other specific duties can’t always be the priority, but we are currently giving a faith effort in which I am confident.

Another comment I would like to make is that the second biggest reason I see staff seeking other employment is due to their jobs becoming increasingly overwhelming. With many funders expecting more and more, and the burden falling on already overworked staff, I fear we will lose well trained and seasoned staff that love their work, in exchange for employers that are more realistic about the strain they put on their staff.



2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

Realistically, I could use at least part time help now to pull everything together. The burden of this would take at least 1 full time person, since I would also have this person to take over doing the community needs assessment, assisting with strategic planning, organizational standard compliance/reporting, logic model completion, ROMA entry of programs who do not allow this with their funds (currently we hire someone part time to do this over the course of several months). The total estimated cost (including fringe/indirect) would be approximately $60,000 per year, plus additional basic support costs for staff related expenses, including training (estimated at $10,000 each year).

There would also be additional expenses staff (includes salary/fringe/indirect) such as additional data collection/research time, training, tracking families, involved in their data collection & input. This is estimated at $59,000 per year. And/or, hiring more staff to take over their some of their current duties so they have more time for reporting/tracking clients/all other duties that will come with this. This could potentially have an additional cost of $70,000 to $140,000, depending on the number of staff needed.

Please note, these are just estimates and the costs could run much higher in the first and subsequent years. KCEOC uses the state associations system; therefore, the cost for this is unknown. No system currently exists in the state that can integrate all the systems utilized by KCEOC.

3. The quality, utility, and clarity of the information to be collected; and

Without help to create a position to be our “Assessment Coordinator,” all we can do is our best and hope we can meet these criteria in the reporting. See additional comments below.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Technology is great, but you still have to have people to input and collect the data.

Comment 66 (Lakes And Prairies Community Action Partnership ; Other/Unknown in MN)

Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

While there are elements that may be helpful, the package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided below that outlines where changes need to be made to reach a level of workability and utility.

The accuracy of the agency's estimate of the burden of the proposed collection of information

The agency's' estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under estimated. Based on our calculations, the burden to our local agency would be 2,360 hours. The cost of this time burden is estimated to be $50,400. Included in this estimate is staff time for training, data entry, and report collection for 4,000 clients annually. Although there is a new online data collection system proposed, this online automated system will most likely not benefit us as a local Community Action Agency. Our agency will in fact need more complex systems that track customers, services, outcomes, given that the current IS survey includes 300+ data elements to report, and the proposed CSBG Annual Report includes more than 1,000, this increased burden of reporting comes with no new resources.

The quality, utility, and clarity of the information to be collected

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level. Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading.

Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and — at the discretion of the states — the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)." It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation. OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and "talk" to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 68 (Louisville Metro Community Service ; State in KY)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

Louisville Metro Community Services (LMCS) recognizes that the proposed CSBG Annual Report is an important step to modernizing an outdated and cumbersome process and applauds the hard work of the network and network leadership to undertake this long overdue task. The utility and necessity of the information contained in the CSBG Annual Report can be best described as a mixed bag. Some items are useful updates; some items are so cumbersome that they will result in reductions of client services to allow just for measurement and reporting of information that brings little value to our analysis and evaluation efforts.

· Module 2 (Agency Expenditures, Capacity, and Resources): The new CSBG domains better capture the work typically performed with CSBG dollars.

· Module 3 (Community Level): Our agency recognizes the importance of reporting the change we create at the community level; however, the proposed community outcomes and their required reporting detail would result in significant negative consequences for our agency, our community, and our larger network. The amount of detail needed to report a single initiative adds no value for our agency. The proposed reporting process would result in having to move direct client service resources into evaluation and reporting and would result in no known benefits for the agency or community. The level of detail required is unreasonable. The measures themselves measure the community in its entirety, meaning the success of agency programs will not be measured by how effective the agency was in creating a new program for a given population, but whether or not the local economy had a good year, in all likelihood independent of the work of any single agency. Much of the requested data is simply census data that would be pulled from the federal government to then report back to the federal government. CAAs are already required to conduct a community needs assessment and develop strategic plans—census data is collected for these purposes already. To then report in this fashion is simply duplicative and an exercise in how well local agencies can fill out forms. It would be far better use of local resources to require Agencies to complete and attach their strategic plans.

This network has underreported in goal two since the inception of the NPIs—making the process of reporting more cumbersome and detailed will not result in improved reporting. Agencies will have to consider whether we want to fund a position to do this reporting or to just do the work and underreport. Our network is about creating positive changes for low-income folks and the communities they live in, not to generate sophisticated reports for the federal government.

Module 3 was developed with little to no input from the larger CSBG network, as will likely be evidenced in the comments OMB receives. Module 3 should be removed from the clearance process and sent back to OCS to revisit in an inclusive way with the CSBG network.

· Module 4 (Individual and Family Level): Section A of Module 4 is not necessary for the proper performance of the agency. Trends and changes in the population served can be identified by comparing Section G reports to each other throughout the year and by comparing this information to already available census data. Under the proposed reporting guidance, when a client has a baby, will this individual now be considered a “new” client? When the children of clients turn 18 and move out into their own households, will they now be considered “new” households? Aggregating this data also becomes problematic, as clients may become duplicated when they move from one area to another. There is no practical utility for this report.

Section B of the report includes new categories that may have some practical value for our agency, including the number of disconnected youth and veterans served by our agency. These aforementioned new categories are a welcomed addition to the Section G report.

Section B of module 4 is by and large a solid proposal for measuring the changes in people’s lives from the work of Community Action. However, there are three major sticking points in these new proposed measures: o The goals for employment lack the ability for CAAs to capture individuals who come to the agency that already have a job and need assistance to prevent or stabilize a crisis that would ultimately lead to the loss of that job. For instance, a CAA may provide childcare (or Head Start services) that allow a parent to continue working. This outcome should be measured, and is an important way to measure the impact of crisis prevention and stabilization efforts undertaken by the agency. O The stability measures are not appropriate or useful for agencies. The first measure has no value—it is simply a count of who was served. The second measure only considers financial stability, and while important, financial stability is not the entirety of stability for folks in poverty. In working with a client who is homeless, the measure of stability may be a stabilized housing situation. For a senior citizen, it may be the ability to remain in his or her home. Neither of these situations, common situations addressed by CAAs, would result in an improvement on the CFPB Financial Well Being scale. This measure could be left in the income and asset building section for those clients this is appropriate for, but should be removed from the stability measures unless others are made available for the vast array of stability outcomes CAAs support. O Section C of module 4 does not seem appropriate to be used on the federal level. This information may be collected and used by local agencies, but does not seem to have value as it is aggregated.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

· OCS’ estimate of a mere 242 hours to implement all elements in the CSBG Annual

Report packet is a severe underestimation. Fully implementing just the community-level reporting and outcome measurements requested would require a full-time position (2,080 hours a year) dedicated to nothing but reporting and measuring outcomes. With serious reconstruction of the community-level outcomes and their reporting, the average burden per response appears more accurate.

3. The quality, utility, and clarity of the information to be collected; and

· Module 2 of the CSBG Annual Report packet, in general, requests information that has practical utility and will be of high-quality due to the simplicity of collection.

· Module 3 of the CSBG Annual Report packet requests low utility data regarding all community-level initiatives performed by agencies. Considering there are 1,035 eligible entities, it seems the lack of standard definitions for something as simple as
“community” will result in data that cannot be scaled or compared.

· Module 4 of the CSBG Annual Report packet, with the exceptions of sections A and D, includes data that has is high in quality and utility for the local agencies. Clear guidance will need to be given to CAAs to ensure NPIs are clearly understood and reported to maintain data quality when aggregated.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

· To minimize the impact on agencies, the community-level outcomes proposed in module 2 should be reworked entirely. Otherwise, the impact is not so significant that major changes should need to be made to the processes already in place to collect data.

Comment 69 (Lycoming-Clinton Counties Commission For Community Action (Step), Inc.; Eligible Entity/Local Agency in PA)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility Each grant administered by STEP has specific requirements, often the outcomes are able to integrate into the CSBG IS, but sometimes they are not. Having the opportunity to insert your own NPIs is important. The information is valuable, but the extent of what needs to be gathered should not over burden the Community Action Agency. The report should NOT be called the CSBG Annual Report – rather it should be called the COMMUNITY ACTION ANNUAL REPORT. CSBG mandates collection of data for all program activities, not just for ones utilizing CSBG funds. Thus calling it CSBG Annual Report is completely inaccurate and makes it confusing!

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information From an agency-wide perspective, the amount of time that needs to be focused on CSBG-related items is intense. Considering the Community Needs Assessment, Organizational Standards, Reporting for CSBG-IS, plus all the requirements and tasks that go along with these items – the overall time burden is at least one FTE staff member. This staff member is often Administrative in nature with a higher capacity to understand the unique inter-related nature of Community Action. STEP’s Management Team made up Chief Operations Officer, Chief Financial Officer, and President & CEO are also heavily engaged in all CSBG-related items. Yet, the funds received through CSBG is only approximately $330,000. When our budget is $24 Million and everything falls under CSBG/ROMA fundamentals, it is a lot of effort related to such a small amount of funds. STEP has recently developed its own online data collection and reporting system through Client Track and invested significantly in it – any changes to this system will be costly. On the State level many of the systems in use cannot be migrated to Client Track due to State agency requirements, so multiple systems still have to be utilized.

3. The quality, utility, and clarity of the information to be collected Many of the indicators are the same or similar. The biggest area that there are issues with is the Community Level Indicators. There is no way to successfully determine percentages for what is being asked with making significant assumptions about how Community Action Agency programs have sole responsibility for community change. This is unrealistic; community change only comes through partnership and collaboration with other entities. Community Action Agencies are often the catalyst for community change, but not the sole responsible party.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology. Over the last five years, STEP has invested significant time in terms of information technology to support the CSBG-IS Report. Over $200,000 has been invested between purchase and training time for the Client Track software deployed in 2014. STEP has had significant challenges with getting State entities able to transfer client information into our system, thus multiple systems are still in use throughout the agency. A new system provided by OCS will only complicate what we are currently doing. Client Track has been developed to mirror the requirements of CSBG-IS on individual/family level and provide Outcome Matrices for each with trackable goal plans.

Comment 70 (M.E.O.A.G. Inc.; Eligible Entity/Local Agency in OH)

Nancy the time and effort for Lou and I to come up with the cost for ROMA next generation would be extensive of itself, but what part I have seen, I would estimate new software, hardware, staff time and training if needed we probably will have around 20% of our 2 year grant of $445,000 if not more. Our cost for the new technology will be the same price as those agencies with bigger CSBG grants then ours. Thus the burden of implementation will hit us harder financially. These are just my thoughts with no facts to base my comments on.

Comment 71 (Maine Community Action Association ; Other/Unknown in NE)

Upon reviewing the proposed new requirements for the CSBG Annual Report, MCAA was disappointed to find that recommendations by the National Community Action Foundation (NCAF), the National Community Action Partnership (NCAP), and state and local agencies were not included in the revised system requirements. Instead, the reporting requirements are triple what the current survey necessitates, forcing CAP's to spend more time on administrative tasks. CAP agencies, state government and the federal government would be better served by streamlined reporting, such as pared-down ROMA NG system recommended by NCAF in their April I 3, 2016 comments. The proposed new reporting standards also lack flexibility, making it difficult for Caps to capture some of their achievements. Paradoxically , while the new standards have more reporting categories, they may not allow an agency to fully capture the benefits of some of their programs. In the case of CAP agencies, one size does not fit all.

The proposed system for information collection will be overly burdensome to our agencies. We share ACF’s commitment to increasing accountability on the part of CAP agencies. We agree that the goal of our services must be to make material improvements in the lives of the families we serve. However, we are concerned the new reporting requirements under ROMA-NG will greatly expand the work of CAP staff without improving the reports ability to measure that improvement.

The current survey requires reports on 374 data points, while the new proposed report includes more than 1,100.Additionally, there are no new resources provided to agencies to cover the cost to compile all of this extra information. This result will not be improved services, but fewer services as staff time is diverted to address reporting requirements.

An additional cost will be implementing data and reporting systems to manage and submit the new ROMA NG reports. The additional expense for software and training of staff are not covered by this initiative. We are very concerned meeting these reporting requirements may require double entry of data, an inefficient and expensive option. We strive every day to maximizing our staff members' time and focus on service delivery over reporting. We would prefer that our staff not be needlessly bogged down in administrative requirements, and that our funding be used where is it most necessary ...helping the individuals that we serve.

MCAA endorses the recommendations brought forward by NCAF and NCAP to improve reporting requirements without making them excessively burdensome to agencies.

We appreciate your attention to these concerns and hope that you will consider them carefully as you finalize the requirements for the proposed ROMA Next Generation system.

Comment 72 (Massachusetts Community Action Agency Planners’ Group; State Association in MA)

The MassCAP Planners’ Group met on Friday, March 25 from 10am to 2pm, in order to discuss ROMA Next Generation materials distributed by OCS. Since the time available was short, the group focused its discussion on higher-level responses, while recognizing that many elements of the materials distributed deserve more detailed exploration.

OVERALL RESPONSE TO THE ROMA-NG INITIATIVE. The Community Action staff involved in the March 25 meeting, while recognizing the complexity of the ROMA-NG program and the need for further work, reached the following general conclusions.

· This initiative is vitally important. ROMA-NG is an initiative of great strategic importance to the Community Action network. It responds to challenges which the network has experienced in measuring its impacts and in communicating those impacts to stakeholders. As such, it is very valuable.

· This is a complex and costly process—and CAAs will need help with the required investments. ROMA-NG will require changes in the orientation and day-to-day workflow of agencies, in the interest of greater rigor in data collection. To the extent that these changes are manageable, they are welcome; however, many will create increased time and cost burdens. As the process of implementation goes forward, agencies will need support in handling new costs, training requirements, and associated conversion issues.

· More time for consideration and discussion would be very helpful. The participants in the discussion greatly appreciated the opportunity to offer input. However, the volume of information provided by OCS and the complexity of some of the proposed changes make it difficult to create a complete response in the timetable allowed. We respectfully request a two week extension (until April 15), which we hope would balance the timetable needs of OCS with the imperative of thorough and responsible review by the CAA network.

· A carefully phased implementation process will be crucial to success. A very complex set of systems changes like ROMA-NG is obviously challenging to roll out over a wide range of diverse organizations. As we know from other systems change efforts, introducing new information and new requirements in a stepwise fashion can help to support adoption and satisfaction. The ROMA-NG program has the advantage of being readily adaptable to “staging” For example,

o The Theory of Change is readily understood, and does not require agencies to change their day-to-day operations. This tool can be provided to agencies quickly.

O Individual and Family NPIs, while they may benefit from some adjustment, incorporate a straightforward set of outcomes measures (as well as a small number which are problematic.) The overall approach used will be readily understood by anyone familiar with outcomes-based management. With the exception of a small number of measures which need further refinement, these NPIs can be implemented on a fairly short timetable.

O The Community NPIs includes a mixture of some measures that are straightforward and easy to use, and some that present serious methodological challenges. While a subset of measures proposed could be implemented readily, some deserve further study or need to be reframed. More time may be needed to work through these issues.

O Demographic Reporting, with the changes as proposed, introduces the powerful but challenging concept of longitudinal tracking. This is likely to require rethinking client tracking and data collection systems for many, if not most, agencies. There will also be a need for significant new investment in technology. Consequently, this is likely to be the element that requires the most training, consultation and technical assistance in advance of implementation.

COMMENTS ON PRESENTATION AND ORGANIZATION. Most of the comments provided in this summary are substantive-- in the sense that they relate to the content of the materials made available for review, rather than the organization of the documents or the language used. However, a number of concerns emerged in discussion which could be reduced by relatively straightforward changes in presentation. These are identified below.

· Clarify that organizations are not required to report on all NPIs. While at present most agencies feel no pressure to report on all of the available NPIs, the level of change required by ROMA-NG has led some agencies to ask if they are “expected” to use all of the new measures. Clarifying that no specific NPIs would be “required,” or seen as necessary to a positive evaluation of agency performance, would be helpful.

· Clarify whether organizations would be asked to roll up reporting on NPIs to include multiple programs. At present some organizations separate reporting on NPIs by program—an approach which might become more difficult using the new standards. Clarifying that multiple program outcomes could be reported under each NPI—and perhaps offering a comment on how agencies could break out program-by-program reporting for their internal use--would be helpful.

· Crosswalk NPIs with services and strategies. A general rubric connecting services and strategies to some possible NPIs associated with them would be of practical use to planners and program staff (although it would need to be understood as a guide for thinking about program outcomes, not as a rigid framework.)

· Crosswalk “new” and “old” NPIs. Similarly, showing a potential framework for relating the new standards to the old reporting framework would be helpful (although clearly it would not be a simple matching process; some “old” NPIs would not translate to the new measures, and many “new” NPIs have no directly corresponding measure in use now.)

· Clarify how the Community level NPIs are linked to the reporting framework for Collective Impact. The use of a collective impact tracking framework is potentially useful, but it is unclear how it would relate to specific NPIs. An explanatory statement would be helpful.

· Provide an overview of how the various elements of the proposed reporting structures fit together. It would be useful to show more distinctly, perhaps in schematic form, how the building blocks of National Goals, NPIs, Services and Strategies listings fit together. More broadly, it would be helpful for planners and other stakeholders to have an outline of the connections between the new measures, the existing systems for reporting using the Annual Work Plan, and the Community Action Plan/Strategic Plan requirements.

· Revisit the numeration of NPIs. While an apparently small point, the numbering of NPIs caused some confusion about how reporting would work. Specifically, there is a logical problem with how the numbered NPIs relate to their subsets; in some cases, the subcategories that would provide an appropriate parallel structure are missing. In addition, it may be useful to adopt a numeration system which incorporates both Individual/Family and Community indicators.

· Make the option of reporting on “other” indicators and services more evident throughout. The group noted with appreciation that an option of “other” was provided in many places throughout the materials; however, it is not consistently present.

COMMENTS ON ‘THEORY OF CHANGE.’ The TOC graphically depicts the essential elements of community action. It embraces the people we serve and reinforces that we are all part of the same community. However, while the language used does articulate the importance of fostering stability and economic mobility as well as the overall health and vibrancy of the community, it does not mention our commitment to reducing poverty. While we do need to add layers of meaning to our discussion of our work in terms of the need to address inequality and share prosperity, we do ourselves, and more importantly, the people we serve, a disservice when we leave out the word ‘poverty’--as it is the fundamental condition we must address in the richest country in the world.

COMMENTS ON INDIVIDUAL /FAMILY LEVEL NPIs. The proposed Individual/Family Level NPIs cover an appropriately wide range of program interventions, and offer a much-improved framework for reporting. It is notable that all of these measures are outcomes, as opposed to the existing NPI structure, which includes many output measures. The long-term benefit of these proposed changes is likely to be significant and positive—supporting increased credibility for the Community Action system as a whole, and allowing for increased comparability with other publicly-funded programs (e.g., Head Start, WIOA).

Specific Observations. While the overall thrust of the NPIs is extremely helpful, a number of specific issues suggest the need for continued refinement.

· A need for interim measures was identified in some domains. This could potentially be addressed using a scales-and-ladders approach.

· Specific concerns were identified with the focus on “Living Wage” jobs. Since access to living wage jobs is largely determined by macroeconomic conditions over which individual CAAs have little control, this was seen as a double-edged sword for reporting purposes. In a related note, the MIT Living Wage Calculator was seen by some as unduly complicated for use in this context.

· The absence of a measure specifically related to gains in income was seen as a weakness. It was noted that income gains are more frequently demonstrable than acquisition of a living wage job, and represent an intuitively meaningful standard which non-specialist stakeholders understand.

· The effort to align measures with those in use in other systems (Head Start, WIOA), was appreciated. There may be a need to further check this alignment, and to ensure that terminology and definitions are consistent. A particular instance noted was Items 3 and 4 under Health and Social/Behavioral Development, which appear to diverge from Head Start performance measures.

· Several NPIs were considered weak in terms of their level of specificity and/or probable consistency across agencies reporting. These included the Item 2 under Health and Social/Behavioral Development, “number and percent of individuals who demonstrate improved health and well-being,” Item 1 under Income and Asset Building, “number and percent of individuals who achieve and maintain capacity to meet basic needs” and the similar “basic needs” measure under Individual and Family Stability. For each of these items, a listing of potentially valid objective standards would be helpful as guidance
(Option Three under Individual and Family Stability addresses this issue.)

· The options offered under Individual and Family Stability raise larger issues related to data collection and analysis. While Option One reflects our shared interest in beginning to quantify the differential impacts of bundled services, it appears out of place as an NPI—this could be profitably considered in conjunction with changes to the Demographics report, since it requires a structured process for longitudinal assessment. (The inference that multiple outcomes are equivalent to “stability” might also be questioned.) In Item 2(b) under Option Two, there is a reference to multi-year tracking, which is a very important goal—but one for which many agencies do not have appropriate data systems. This item is followed by a discussion of cross-tabulation of outcomes with numbers of service contacts, which is also likely to be a technical challenge for a large proportion of agencies. Finally, there is a problem with outcome statements under both options in that it is unclear what the denominator would be for “number and percent of individuals”. Overall, these NPIs raise important questions for which Community Action needs to develop answers—but the technical framework through which they can be answered is, at best, still under development.

COMMENTS ON COMMUNITY LEVEL NPIs. The development of an ambitious set of Community

Level National Performance Indicators, and the introduction of the Community Level of Effort Progress Tool, are welcome changes which have the potential to create powerful statements of impact. They support the strategic imperative of elevating the visibility of the community work done by CAAs. Flexibility is key here; it is important allow CAAs to describe the work they already do, and not to tie them to a set of indicators which are unrealistic or unduly rigid.

Specific Observations. While the strategic direction represented by the Community NPIs is very positive, there are significant issues with some of the proposed indicators.

· There are fundamental problems with indicators which use changes in population-level ratios as the measure (e.g., “percent increase in children who are kindergarten-ready” or “increase in adult literacy rate.”) These measures, which are appropriate for large-scale public health interventions and efforts of similar scope, are unlikely to produce useful information in the context of a single agency’s actions over one year. The group is aware of the OCS guidance which allows for individual definition of “community”—however, this is unlikely to eliminate the problem. If “community” is defined as a service population (which would be necessary in most cases in order to report meaningful outcomes), the measure is essentially reporting on the success of a particular CAA program; this is information that fits more appropriately under Individual and Family Level NPIs.

· Conversely, most of the NPIs which use scalar measures (e.g., “number of jobs created”, “number of safe and affordable housing units developed”) are workable.

They may require further definitions or clarifications (some of which are noted below).

· For a number of Community NPIs, there may be need for further guidance on the conditions under which it is appropriate for agencies to report that jobs, housing units, or other resources are “maintained”. This type of measure can lack meaning unless there are specific standards which ensure that the reporting CAA is actively engaged in activities which support “maintenance”.

· In a number of Community NPIs, the phase “number and percent increase/decrease” is used; this is generally not meaningful unless the denominator in question is defined. In most cases, these NPIs would be clearer without the “percent”.

· The organization of Community NPIs related to “Civic Engagement and Community Involvement” is confusing. The intent to separate contributions by “people with low incomes” and others is understood, but splitting them under two National Goals creates a conceptual barrier for the reader. The indicators listed under National Goal 2 (non-low-income contributors) are generally weak (they represent number of donors, number of volunteers, and the number of people who come to meetings, none of which may be very meaningful.) However, the Indicators under National Goal 3 (low-income persons) are clear and succinct.

· The introduction of the Community Level Efforts in Progress Tool was seen as a positive step. This tool will require additional training and technical assistance for agencies, but it provides an initial effort to extend the agencies’ reporting horizons to efforts which stretch across multiple years. There was some confusion, however, about how (and whether) this tool would be linked with specific community NPIs.

COMMENTS ON INDIVIDUAL/FAMILY STRATEGIES AND COMMUNITY LEVEL STRATEGIES. The listings of strategies and services for both Individual/Family and Community are extensive and largely non-problematic. There was, however, some need for more explanation of how this data would be reported, and whether it would be integrated with NPIs.

COMMENTS ON DEMOGRAPHIC REPORTING. For the most part, changes regarding the specific data elements to be collected for demographic reporting were seen as non-problematic. However, the approach to measuring CAA population changes against a baseline was seen as seriously flawed --for reasons which relate to both basic statistical methodology, and to the practical limitations faced by individual agencies. The following are some key points made in the discussion.

· The goal of gathering information on the experience of CAA consumers over time is extremely important, and should be pursued energetically. The network is interested in conducting long-term evaluations but it will require a more fully developed research model.

· The two-point process shown here will not produce meaningful data, for reasons including the following.

O To create meaningful group comparisons, the two groups being compared must be composed of the same individuals (e.g., a cohort.) To create a cohort composed of ALL of a CAA’s consumer base, data systems need to exist which will consistently preserve a “starting point” data set and then match it with a “second point in time” data set. Most client tracking systems now used by CAAs overwrite data as it is updated, making this close to technically impossible. While it is possible to create a system which can produce this result, it is very likely that such a system does not exist now.

O If such a system is to be developed, the data points being compared need to be thoughtfully selected; the appropriate time frame also needs to be discussed. While we may be interested in employment and education changes, for example, tracking family composition and age may produce mostly “noise.” It is also unlikely that real change will be seen, for most households, in one year. Multi-year tracking will be needed, but it brings with it additional technical complications.

O A longitudinal tracking model will require broad operational changes for many CAAs. Currently, many CAA consumers receive services only once a year. More generally, CAAs may not have systems in place for re-assessment. Changes in this regard could be very positive, but they will take time and money.

Comment 73 (Maui Economic Opportunity Inc. ; Other/Unknown in HI)

Will there be education for the funders- CSBG is so complicated and the funders may not understand why the statistics changed and what changed. First of all, funders need to understand CSBG and what the purpose of CSBG. Also, every grantee uses the funding for different purposes.

Are the stories going to capture the true essence of poverty and what multiple barriers need to be addressed to get individuals and/or families into self-sufficiency?

What is self-sufficiency (this is a question for both ROMA and ROMA Next Gen).

How will there be consistency in success stories from one grantee to another?

Data collection will be more time consuming with collecting data on multiple years. Meeting all of the requirements of the proposed data collection and reporting will be very difficult without a centralized collection system; however staff cannot be burdened with being responsible for capturing data in multiple databases. Could OCS provide funds to invest in a system that is able to export data from other data systems for the CSBG reporting (Rhode Island has such a system)

Extend the rollout date out by another year. The change is significant enough to allow more time for implementation.

Remember that Hawaii is also a State with certain conditions that are not present on the mainland. When doing a pilot, include an agency from Hawaii.

Comment 75 (Michigan Community Action ; Eligible Entity/Local Agency in MI)

The collection of information does not meet the threshold of necessity or utility. The package is costly, burdensome, provides useless information, and requests data on outcomes that are beyond the capacity of the Community Services Block Grant (CSBG). OCS' development of CSBG Performance Standards has been a model of positive steps for insuring accountability and capacity to maximize the potential of the CSBG. Changes to the CSBG Annual Report are unnecessary.

If there is a need for additional data and information on CSBG programs, an alternative approach to changing the CSBG Annual Report would be to commission an annual research study to measure the impact and effectiveness of typical CSBG interventions. Most CSBG grantees could export a wide variety of data that would allow experienced researchers to answer narrowly drawn research questions and to respond to data needs from OCS and other stakeholders.

Address the unique issues of Public entities that receive CSBG. Public agencies (sub-grantees) that sub-contract most or all of their CSBG funding to other service providers may face added challenges of requiring their grantees to collect data in accordance with ROMA Next Generation requirements since it may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet the reporting requirements. This challenge is not addressed in the proposed reporting package. In Michigan, many Public entities work with faith-based providers and local cooperatives. These neighborhood based organizations do not have the resources to report at this level.

Question 2:

The accuracy of the agency's estimate of the burden of the proposed collection of information.

The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

The burden on local agencies is not impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems and states routinely pass on reporting burdens to the local level because of their own financial and technical constraints.

The agency's' estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under estimated. Based on our calculations, the burden to our local agencies in Michigan would be 431 hours per sub grantee. The cost of this time burden is estimated to be $1,437,385 for Michigan. This does not include burdens on the State of Michigan or Michigan Community Action. These additional hours are estimated to be at least 231 hours for a cost of $26,565 (training, data clean up, system design, working with consultants, etc.).

There is no reduction in burden for local agencies by the online data collection system described in the by OCS. OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and - at the discretion of the states -
the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)." It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely adversely impact local Community Action Agencies (sub-grantees) with more state imposed deadline and data requirements.

Question 3: The quality, utility, and clarity of the information to be collected. Maintaining a flexible data reporting system and engaging researchers to mine the vast trove of data already available is a more cost effective approach than increasing the data provided through the CSBG Annual Report. As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels.

Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide or regional. For example, comparing overall percentages of community indicators (e.g. homelessness, poverty level) to individual local agencies outcomes is misleading

Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed.

Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point.

First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency.

Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes.

Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

I. OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and - at the discretion of the states - the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)." It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and "talk" to one another to produce reports that meet the needs of local agencies, states, and OCS.

To truly minimize the burden of the collection of information, DHHS and HUD and DOE should take 12-24 months to review their combined data requirements of shared grantees. Utilizing one data system for federal programs that impact low income families would allow for more robust data alignment and outcome data without the need to ask any grantee for additional information.

Comment 76 (Minnesota Community Action Partnership; State Association in MN)

Question 1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

1. Module 1: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported and creates a misuse of the Organizational Standards. Expecting constant perfection is unrealistic and does not have practical utility for local agencies or the State CSBG Lead Agency. As proposed, the single metric the State Agencies will report on will be the number of eligible entities that meet 100% of the CSBG Organizational Standards. To increase utility of the information, State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

2. Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency. This is not necessary for agency performance, and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will be skewed by large population centers and decrease the practical utility of the data even further. Furthermore, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems. In addition, as many local agencies serve families over time, with no definition of “new” being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years (e.g. Early Head Start, child care, after school programs, youth programs, job training, WIC, food boxes, LIHEAP, etc.). In Minnesota, was this to be analyzed as statewide aggregate data the volume of client data from the LIHEAP program in the Minneapolis/St. Paul metropolitan area would dwarf all other data sources and our statewide aggregate number(s) would shed no light on Community Services Block Grant activities?

3. Module 3: Remove the Community Outcome Indicators that include percent or rates. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

4. Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work. While CSBG eligible entities are engaged in such efforts, privileging one approach is inappropriate and unnecessary given the broader range of options for doing community-level work. Collective Impacts should be removed and included in training and technical assistance endeavors as a potential local option.

5. Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the eligible entity’s work. One option duplicates data already collected and the second uses a tool that defines financial stability based on a construct the uses assumptions drawn from the general population, not low-income individuals and families specifically. Because financial well-being means something very different for the low-income population in comparison to the general population, use of the scale will produce data of limited utility. 6. Address the unique issues of Public entities that receive CSBG. In Minnesota, we have no public entities.

7. Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network to commit resources for ongoing training, technical assistance, and general capacity-building supports consistent with private sector approaches such as Continuous Quality Improvement, Lean and Six Sigma.

Question 2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.

At best, it is not clear how the burden for local agencies is decreased by the online data collection system described by OCS. At worst, the proposed system may never be implemented irrespective of any value or virtues it may have had. In the absence of a system’s RFP specifications, a system budget and a formal project governance structure and timeline it is difficult if not impossible to assess the probability of successfully implementing the proposed system. Consequently, if one time burden analysis is to be conducted with one hypothetical scenario of successful implementation, than it seems prudent to also perform a second hypothetical scenario time burden analysis with an assumption of no new system. In the event of failed implementation, substantial additional state and federal level resources would have to be diverted away from capacity building which supports high-quality programs and be spent on data collection.

Question 3: The quality, utility, and clarity of the information to be collected.

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

· Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

· Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

· Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

· As noted above, OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

· OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 77 (Minnesota Valley Action Council, Inc. ; Eligible Entity/Local Agency in MN)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

A national report serves many roles. It provides a picture of what agencies are doing with grant money, it demonstrates impact of the money being spent through client reach and results, and finally, it should offer up direction for where the network is going. Everyone wants to demonstrate the good work we do and results we are getting. Because Community Action is so diverse, this is difficult to do with number in silos. Likewise, a utilization (or impact) report, does not improve the performance of an agency. In this format, the practical utility is lacking. Reporting the number of strategies does not provide usable data – a strategy may take place once or over a year or more, may require one staff or fifty. This is unusable data. It also does not provide usable data to anyone at the national level, as there is no apples to apples comparison of services that allows to tell success stories. Many of the changes would be better served to be part of an agency local plan / community needs assessment rather than reported nationally. A report does not improve performance or function – this is for the assessment and strategic planning portion of ROMA.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

Based on the estimate of 164 hours for annual reporting, this is an eight-fold increase from our current time spent on reporting. However, we believe this initial estimate is quite low. For staff training, additional data acquisition, database maintenance, data transfer, and report entry, and CSGB Standard reporting, we estimate this will take realistically 400 hours of staff time. As an agency that has 9 offices in as many counties, staff training and data acquisition is quite extensive. Utilizing a conservative $25 per hour, this is $10,000 just to report and produce this document. This is, in effect, a cut in funding and therefore a cut in services for a requirement that does little to nothing to improve the quality of services.

3. The quality, utility, and clarity of the information to be collected; and

The information requested (specifics provided below) oftentimes to lacks reliability and validity. There are many terms left up to interpretation that eliminates the ability to generalize or apply the data across the country. Many of the data elements are subjective or difficult to compare accurately. Attempting to report “Collective Impact” reminds us of Al Gore’s fuzzy math in 2000. The attempt to have agencies report any and all potential impacts they may have as an individual agency as well as part of a collaborative is, at best, a guess. I do not believe that you are going to get the information you seek with this report.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Go back and reconsider why are we collecting this data and what are we trying to do with it. If we are trying to define impact and outcomes, then this is a “miss” for a report. Everyone says “if you see one CAA, you’ve seen one CAA” – the same goes for reporting. You can’t silo what we do in 1000+ different agencies into these overgeneralized data points with subjective definitions. While allowing agencies flexibility in choosing how and when to report, you have completely invalidated the report. The inevitably encourages agencies to omit poor outcomes and include good ones, or adjust geographic areas to infer greater impact of a program. I don’t know how automating will fix this. The burden of reporting is on the sheer volume of the report. If it takes 76 pages to describe how to do a report, something is wrong. CSBG is a mere 2.1% of my agency’s funding, but by far has the most excessive reporting requirements if ROMA NG goes into effect. This reporting pushes us to use this small amount of flexibly funding and use it to report on itself. Go back to what you are trying to tell with this data and start over.

Comment 78 (Missouri agency Inc. ; Other/Unknown in MO)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

I agree that the collection of data is necessary to substantiate the work that we do, and prove that our work is valid. I also feel that the community action network in Missouri is ready to begin the process of ROMA Next Gen, but I am worried that many states are not up to the task yet, which can potentially be very harmful to community action, overall. If the other states do not have the resources needed to collect the required information, how will that affect the entire network when legislators and other officials review the work we have done?

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information.

Considering that the current IS report includes 300+ data elements to report, and the proposed CSBG Annual Report includes more than 1,000 data elements, I believe that OCS’s estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is somewhat under estimated. Based on our calculations, the burden to our agency would be 396 hours. The cost of this time is estimated to be $8,740. Please note, this estimation does not include the T/TA that will be needed for our employees to understand how to complete the new report.

3. The quality, utility, and clarity of the information to be collected.
If the proposed number of general indicators were decreased, for example, 2-4 general indicators for each of the domains, then the aggregate data collected from the entire network would be more focused and refined. Which would then be much more meaningful data to present to legislators and funders? To show more localized impact, agencies should still be allowed to track other outcomes using local indicators more specific to their unique programs and communities.

For more clarity in the information collected, I feel that the data would have more impact if the indicators were worded similarly to other programs, such as Head Start and HUD. If all programs reported using similar indicators, then, community action would have more impact on a national level.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

In Missouri, we have been using a state-wide data system for many years. Several other states do not have this resource, which is absolutely necessary for collecting accurate data. The proposed on-line automated system is a nice thought, but much of our agency’s reporting comes from many different data tracking systems. Our other programs, Head Start, Women’s Wellness, Housing/HUD, and Weatherization all have their own systems. Unless OCS can develop a system that can be used in conjunction with all of the other systems, the proposed on-line automated system will not work for local agency. Therefore, the burden of the collection of information at the local level will not be decreased in any way. It is still going to be a large and expensive burden on our agency.

Comment 79 (Missouri Community Action Network (Can); State Association in MI)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The overall concept of the information to be gathered for the purpose of validating the work of community action is absolutely necessary. The Missouri Community Action Network is prepared to meet new requirements of information collection and appreciates the work that has been done thus far. Although we as a state feel comfortable with the idea of ROMA Next Generation, we are concerned with the overall impact to the network if other agencies and states do not have the resources or training to collect information within the proposed framework of indicators and requirements.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

The time burden seems slightly underestimated. The level of expertise to conduct some of the proposed data collection and analysis is heightened as well. Additional skill sets may be needed, which would increase the overall cost burden associated with the CSBG Annual Report. The investment in the infrastructure for human resources, expertise, and systems is not present currently to support the proposed collection of information.

3. The quality, utility, and clarity of the information to be collected; and

Quality: The proposed set of indicators seems to focus more on quantity vs. quality. Our recommendation would be that the number of general indicators is decreased (although we know not all agencies must report on each one). If a smaller number of indicators were used to aggregate data to the state and national level, this may be more meaningful to decision makers. Agencies could then use local indicators to track additional outcomes based on their unique programs (For example, if there were 2-3 indicators relating to each of the domains of CSBG to provide
meaningful aggregated data). The local analysis expectation could then be strengthened to fuel the ROMA cycle within the evaluation and assessment portion.

Utility: To the point above, we feel the information would be most useful in fewer, more meaningful indicators that can show national impact of community action in a more concise way. Also, it would seem appropriate to use the language of other indicators that are already established through Head Start or HUD to promote further consistency.

Additionally, our statewide network would request access to the local, state, and national comparison data after compiled to use and share locally.

Clarity: We respect the strengthened focus on community and believe it is necessary. However, many of the proposed community indicators could be confused with individual or family level indicators. The meaning tends to be similar, as well as the unit of measurement, while the wording is different. Examples include:

· Community Level, Education #9 = Family Level, Education #8, 9, 10.

· Community Level, Education #5 = Family Level, Education #7.

Next, there are terms we have identified that would need clarified definitions before moving forward to ensure the network is collecting similar information:

· Low-income

· Community

· Living wage

· “New” in characteristics report

Lastly, the variation of percentages, numbers, and rates seems confusing and inconsistent. We understand the need to examine percentages when looking at an increase or decrease and numbers are helpful in other circumstances.

The way some indicators are written causes confusion on what is being counted or measured. The rates should most definitely be examined, but possibly just in the community needs assessment to be analyzed for multi-year trends.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

We consider ourselves fortunate to have a statewide data system capable of producing an accurate unduplicated count and state CSBG administrators who value this system as a priority of CSBG discretionary funding. Since 2004, Missouri has invested more than $1.5 million in our statewide system, excluding agency staff time. Many states do not have resources or funds allocated for this. We are increasing our efforts around consistent use of the system and upgrading to a new version to prepare for such changes in reporting. Training and technical assistance (T/TA) will also be necessary to ensure counting with consistent definitions and systems/procedures for efficient data collection.

We also know the web based data tool in the community commons platform will become an increasingly valuable asset. Three states currently have a state level tool to gather community and tract level data from additional state level data sources. The indicators that are available through these tools nationally and at the state level should be considered in relation to the proposed indicators.

Module 2 – Agency Expenditures, Capacity, and Resources: (pages 23-27)

Overall, we agree with the need to improve reporting and data collection in community action. However, it should be manageable with the resources agencies have available. This is not to say that requirements should not be put above where many agencies are to obligate improvement, but the improvement should be possible and with adequate investment via funds and resources. We know this work is important and do not want the inability of some to meet the requirements or provide adequate data to cause community action to appear ineffective. The NPIs should be able to be aggregated at the national level, be meaningful to decision makers, and show the effectiveness of community action.

Comment 80 (Montgomery County Community Action Development Commission; Other/Unknown in PA)

I am writing to provide comments regarding ROMA Next Generation:

· We believe that the amount of data that would have to be collected is burdensome and excessive.

· The cost of purchasing and implementing the new system would be substantial and require a significant amount of additional resources. The start-up costs, training and additional staff time, etc. could exceed $100,000.

· If we were required to pay for the new system out of current CSBG funds it would adversely affect the number of people we help and services we provide. This in turn would decrease our impact on the community.

· The challenges associated with designing an integrated local system that can produce valid client tracking data as it relates to progress, analyze situations at various points in time and measure community impact may be insurmountable at this time.

· In our estimation, the time needed to implement the new system from the beginning to fully operating far exceeds the proposed start date of October 1, 2017. We believe it could take at least 18 months if not longer.

Therefore, we ask that you please you reconsider how the new system will impact all Community Action Agencies, the people we are trying to help and communities we are working hard to change prior to implementing ROMA Next Generation. Thank you,

Comment 82 (National Association For State Community Service Program ; Other/Unknown in N/A)

Thank you for the opportunity to comment on the proposed Community Services Block Grant (CSBG) Annual Report. NASCSP appreciates the ability to actively participate in this process through our current cooperative agreement, Performance Management System Development and Data Collection Analysis and Reporting for the CSBG Program, with the Office of Community Services (OCS). As a training and technical assistance provider and membership organization for State Administrators of both the CSBG and Weatherization Assistance Program, NASCSP continues to applaud and actively support the federal investment and national commitment to reducing poverty in our nation that the CSBG and weatherization programs represent.

As part of this cooperative process, NASCSP has convened, participated in, and promoted a wide range of opportunities for engagement during the process of drafting the proposed CSBG Annual Report and served as a conduit for soliciting and aggregating CSBG Network feedback. We appreciate that this is a collective, network-driven process that has been rich with engagement and dialogue. The timeline for distribution and comment between the initial release of information and the start of the 60-day comment period took place in a relatively short timeframe. This has left challenges and a remaining need for feedback, revisions, and areas of compromise to ensure a strong final product for the three year clearance of the CBSG Annual Report. We have reviewed the proposed CSBG Annual Report to identify both the strengths and potential weaknesses. Our feedback is provided below and addresses the key questions posed by OCS, as well as module specific feedback.

Practical Utility, Estimated Burden, and Quality and Clarity of Information to be Collected As we work with all levels of the CSBG Network to revise and strengthen our performance management framework and modify our performance evaluation system, we believe that the proposed CSBG Annual Report will ultimately help showcase robust results achieved by the CSBG Network for individuals, families, and communities with low-incomes. The CSBG Annual Report will help us meet the latest reforms and expectations for performance management across the government and private sector in order to focus on analyzing results and using the data to improve programs and the outcomes achieved. ROMA Next Generation introduces new elements into the CSBG Annual Report to support the analysis of services and strategies provided and the impact of these services and strategies on individual, family, and community change.

The modified CSBG Annual Report will provide clearer descriptive data that will demonstrate outcomes at the individual, family, and community levels, and foster improved data analysis at every level of the CSBG Network. We recognize the significant improvements within the proposed CSBG Annual Report, namely the addition of the Strategies and Services reports, the expanded section on community level work, modernized language and indicators, and the intent to automate State level reporting in OLDC.

At the same time, NASCSP urges OCS to recognize and take into account the critical training and technical assistance elements necessary to successfully implement the CSBG Annual Report and capture quality, meaningful data. Listed below is a high level summary of areas of concern as related to the CSBG Annual Report and is intended to serve as an “At-A-Glance” version of this letter. These concerns are elaborated upon in the next section of this letter:

1. Implementation and Timeline

The completion of the CSBG Annual Report will require substantial investments in staff time and resources at the Federal, State, and Community Action Agency (CAA) levels. In order for this important endeavor to succeed, it is essential for OCS to modify the projected implementation timeline to allow for the necessary training and technical assistance and systems upgrades. It will be necessary to provide significant training to staff at all levels of the CSBG Network to ensure all understand how the different pieces work together and how to complete the various reporting forms included in the new report. It is also critical to allow sufficient time for the CSBG Network to evaluate not only the content, but also the system for collection (either in the On-Line Data Collection (OLDC) system or other system), which has not yet been developed. We expect this training will be needed on an ongoing basis and not just one time at implementation.

2. Estimated Burden

NASCSP recognizes that the burden estimate is an average estimation across States and CAAs, and therefore may either over or underestimates the time and cost burden, depending on variables existing in each State. While we support the revised/automated forms, the eventual creation of an online data collection system, and agree they will yield efficiencies in the States’ processes for completing the annual report (which replaces the IS Survey), the new process also increases State accountability and places new demands on systems and software. For example, the capacity of the system in terms of interactivity and functionality for data upload, storage, and usability is an unknown element in the estimations for user burden. Additionally, the burden on States and CAAs is compounded by barriers to sharing data from other federally funded programs (e.g. Head Start, WIC, HUD, SNAP, WIOA, etc.), because few strong “bridges” currently exist at the Federal level and legal boundaries prevent data sharing.

Because the content (in the form of the CSBG Annual report) is still in its draft form, it is and will be difficult to accurately project costs and time investments needed to implement the final format. Given that both the content to be collected and the processes by which the data will be collected are still under conceptual development, making an accurate burden estimate is difficult as it is dependent on certain assumptions about capacity and implementation. With these outstanding elements, it is extremely difficult for State offices, CAAs, and the Federal government to accurately estimate the true burden of this system, the CSBG Annual Report, and a revised data collection system.

The current CSBG Act, passed in 1998, establishes requirements for OCS, States, and Eligible Entities to collect data and report on the planned and actual uses of CSBG. Each state must submit a report on performance, expenditure of funds spent by the state and eligible entities, delivery of direct services, characteristics of clients served, and training and technical assistance provided to eligible entities. This has been accomplished using the CSBG IS Survey. NASCSP commends OCS for the evolution of the CSBG IS Survey into the CSBG Annual Report and for taking steps to ensure our CSBG Network is fully prepared to move forward in the ever-changing work to end poverty; however, the modifications discussed in this letter are critical in ensuring the CSBG Annual Report can be successfully implemented. Our recommendations include the removal of the New Characteristics Report, simplification of the community measures, and addition of developmental measures. These revisions will greatly alleviate the burden of data collection at all levels while still meeting the requirements of legislation and launching the CSBG Network forward into a learning culture rather than compliance and reporting culture and further the goals laid out in the Act.

NASCSP Feedback – FRN #1 OMB PRA Clearance, CSBG Annual Report

NASCSP urges OCS to extend the proposed implementation timeline to ensure the CSBG Network’s capacity to accurately report on their efforts and outcomes. We acknowledge that time is of the essence; however, in order to fortify administration of CSBG and best document the performance gains of this CSBG Network, considerable revisions and extensive vetting are essential to determining what can realistically be accomplished and done well.

Comment 83 (National Community Action Foundation ; Eligible Entity/Local Agency in N/A)

Thank you for considering this response to the Federal Register notice # 2016-14229 on behalf of the National Community Action Foundation, which represents the Community Action Agency [CAA] network before Congress and federal agencies. The notice solicits reactions to the proposed requirements for the form and content of a mandatory annual report to HHS by State CSBG grantees. It would be based on reports to states from their local sub-grantees.

Our perspective is informed by:

1. Our staff and our Director’s participation in OCS’ intensive and highly participatory working groups over three years;

2. Our governing Board members, all of who are executive directors of CAAs that are elected by their colleagues; and,

3. Over one hundred in-depth conversations with Community Action Agency leaders during the current year alone.

We remain deeply disappointed in the proposed report and in the expanded management information system that it will require for data collection. In April, we responded to the OCS Informal Request for Comment by proposing direct and clear changes and alternatives. HHS has adopted only a few of the changes we suggested. The scope of and elements in the report are similar to the original draft, and there is only minor improvement regarding the extraordinary increase in the paperwork burden this proposal creates. Changes that might enhance local agency capacity and free-up resources for information that measures CAAs local results were not included. Our colleagues at the Community Action Partnership also recommended changes, all of which we supported, but few are included in the revised proposal.

Before outlining our extensive concerns, it is critical to note that the proposal does not accommodate the statutory requirement that OCS allows states to adopt other comparable systems. The proposed templates and the online federal interface, or “OLDC,” into which they National Community Action Foundation ROMA NG Final Comment will be loaded when complete will apparently allow no other format or approach to dovetail with the OCS data system or other state reports. The elaboration of the system elements, right down to the details of the local project management tracking, suggests OCS is not prepared to accept alternative reports proposed by a state.

Section 676(b) (12) of the CSBG Act does not require states and eligible entities to use ROMA. It requires the use of ROMA or an alternative performance measurement system (and, in fact, the proposed annual report document makes reference to such an alternative system). However, the proposed CSBG Annual Report form does not allow for reporting annual performance of a state and its eligible entities under any alternative system. Indeed, the statement in the Federal Register comment request that the proposed CSBG Annual Report incorporates Results Oriented Management and Accountability as “required in Section 676(b) 12 of the CSBG Act,” completely ignores the fact that the Act specifies that states may use alternative performance measurement systems. This draft framework is not “as required” unless it can be expanded to include a system, including a federal data management system that will accept state report data.

While appropriation language adopted for 2016 is different from the statute, it was inserted to give OCS the authority to issue the CAA Organizational Standards with which the network has arrived at consensus. It states that “The Secretary shall issue performance standards for nonprofit organizations [entities] receiving funds from State and territorial grantees… [Which] shall assure the implementation of such standards prior to September 30, 2016 [2017]?”

The CSBG Act states that each State:

Shall ensure that all eligible entities in the state participate, in a performance measurement system, which may be a performance measurement system for the Secretary facilitated development… or an alternative system that the Secretary is satisfied meets the requirements of [678E9b)]… and that each state and all eligible entities in the State…will participate in the Results Oriented Management and Accountability System CSBG Act section 678E(a)(1)(A), 42 USC

National Community Action Foundation ROMA NG Final Comment

9917(a)(1)A) and CSBG Act section 676(b)(12), 42 USC 9908(b)(12).
Further, §42 USC 9917(b) states that HHS:

Shall facilitate the development of one or more model performance measurement systems, which may be used by States and by eligible entities to measure their performance in carrying out the requirements of [the CSBG Act] and in achieving the goals of their community action pans.

§42 USC 9917(a)(2) further requires that:

1. Each state “annually prepare and submit to the Secretary a report on the measured performance of the State and the eligible entities in the State;” and,

2. The annual report that HHS is required to submit to Congress must include, among other things, “a summary of each State’s performance results, and the results of the eligible entities, as collected and submitted by the States…and any additional information that the Secretary considers to be appropriate to carry out [the CSBG Act]. If the Secretary informs the States of the need for such additional information and allows a reasonable period of time for the States to collect and provide the information.”

Acknowledging that the proposal does not accommodate the statutory requirements of §42 USC 9917(a)(2), the following comments demonstrate the extent of further concerns regarding the value of the proposal, and its efficacy as a tool for either “telling the CSBG story” or as the toolkit needed by local agencies to identify and monitor the outcome of their work. We anticipate that the leaders of individual agencies will be estimating the burden of time and cost as part of their own comments.

We have organized the overview section of the response as follows:

1. A summary of our concerns;

2. An explanation of the kind of information system and reports that would work far better as a performance management system; and,

3. A final summary of our proposed next steps in updating the ROMA system.

A second section contains a detailed section-by-section analysis that outlines our concerns.


OVERVIEW

The Community Action network and NCAF share the Administration’s commitment to strengthening a performance-driven culture, including the development of a better system to track the way CSBG-supported investments and projects are meeting their goals. We also agree that the state and national reports must include better descriptive data about CSBG network activities and participants. Furthermore, we agree that such data must also clearly demonstrate impact and results.

The OCS Dear Colleague Letter, dated June 17, 2016, described the intent of the proposed system:

This data will serve as a basis for program analysis at the local, state, and Federal levels to increase understanding of what combination of services or strategies produce the best outcomes for specific populations, family types, and communities.

The new CSBG Annual Report should be about continuous quality improvement at the local, state and Federal levels rather than reporting for reporting’s sake. We do not believe most of the new sections of this draft system serve the intended purpose.

Our five major concerns, discussed in depth in the second section, are:

1. The investment in the proposal’s required reporting will be wasting limited CSBG resources if done before changing CAA data storage and integration systems. The proposal requires that limited CSBG resources, integral to community programs, be invested in system changes that will not improve results management.

National Community Action Foundation ROMA NG Final Comment

The software OCS will develop for states to file reports does not include tools for local agencies to collect and integrate data into individual, family or project files. The promised “OLDC” for filing state reports is not the system to create and organize the data, in local CAA databases, that agencies and states need to track their performance in achieving annual and strategic goals.

Yet to be developed local data systems that can produce clean, client-linked information will take much longer to build and will be prohibitively expensive. Yet, such data systems remain the cornerstones of results-oriented information, and acquiring them should be the priority for resource allocation in the coming years. No national or state report can be valid if it is based on the poorly-integrated information now stored in overlapping federal and state software systems holding client and project files as required by CAAs’ multiple funders.

2. The federal requirement for community project-level datasets is an inappropriate and unprecedented intervention in local agency projects. The scale of the draft mandated report is vastly larger than the current system.

a. In Modules 2-4 there are 1,134 potential items compared to about 340 in the present system. This does not include the data items left empty for agencies to fill in with information that is “other.” Module 1 requires lengthy state narratives including the results of local improvement efforts, which will require local narrative reports.

b. In Module 3, the report requires entry of detailed community level project management reporting for in-progress initiatives. In Module 1, the report requires extensive state evaluation reports [up to 2,500 words] on local organizational standard implementation. We support the opinion of the Community Action Partnership that the use of area demographic data and percentages of such data are unsuited to CAA project goals.

National Community Action Foundation ROMA NG Final Comment

3. Much of the proposed new content will not produce data that can be the basis for any evaluative analysis at the local level, much less when aggregated at the state or federal level with other agency data. In the second section, we identify those data with little value and the details of our concerns of their requirement. Unfortunately, the sections that are only slightly changed will continue to be populated with the often-duplicative “snapshot” data that have been the basis for the first generation CSBG

• Later, we also provide detailed justifications for eliminating the “new participant” report.

4. The proposed rollout schedule is unrealistic. We agree that the incrementally updated characteristics and expenditures reports can take the place of the current forms by FY 2018. If the promised development of the state plan submissions and tracking system, the “OLDC,” is complete by summer of 2017, then we can imagine that a pared-down version of Module 1, the state administrative reports on plan implementation, could be in place by FY 2018. However, OCS has yet to prepare definitions and guidance for many of the new terms and newly required report elements. OCS should define new terms prior to final network review.

5. We support the recommendation by the Community Action Partnership that the new proposal should restore and reflect the CSBG goal to increase the capacity of CAAs to fight poverty.

National Community Action Foundation ROMA NG Final Comment

Our Alternative Performance Management System

A new generation of ROMA can, and should, proceed with significant changes from the current draft. Our concerns about cost, relevance or efficacy can be resolved by:

1. Reducing the number of required components, including the services list, the new participant report, and the stability measures and tracking measures, 80% of the Community impacts items, and offering these proposed elements as a voluntary CSBG toolkit.

2. Postponing the implementation of the newest elements while testing updated complex information systems.

3. Investing in CAA-centric information systems and practices that, in 5 years, can equip local agencies to establish targets and track customized, quantitative indicators of success for the work they have identified as top priorities in their strategic plans. Simplified state reports would include the progress of local agencies toward their major strategic goals. Narratives would present what the goals and targets were in the report year and provide examples of the indicators of results.

4. Such reports would derive state and federal reports from the data CAAs need to measure the success of the year's projects. Narratives should accompany every priority project. This would replace the generalized individual, family and community national measurement with clear indicators of progress, which are appropriate to the mission of Community Action, and reflect its diversity throughout the nation.

HHS is certainly familiar with the problems of interoperability in data systems because of its recent experience with developing health databases and integrating them with some human services information at the state and federal level. It would be very helpful if HHS were to devote some of its Medicaid-funded technical data integration resources to designing data warehousing for all programs that coordinate multiple resources toward a single purpose.

Instead, ROMA NG transfers the burden of creating such a data warehouse to CAAs, though such a task has yet to be mastered by the federal government. In fact, federal agencies have not yet created the interoperability that allows data exchange between programs run by the same agency. For example, many Head Start programs refuse to share client data files with the CSBG data managers in the same agency, citing federal policy. Suffice it to say, there are also no interoperability nor seamless data exchanges between separate domestic agencies at the federal or state level.

High-impact, large CAAs have the greatest data complexity. The smaller the number of funding sources and participant population, the easier it is for an agency to integrate program and client files that have been collected and formatted for those funders. Conversely, the greater the complexity of funding source, partnerships and general capacity of an agency, the more difficult it is to design a data system that integrates and organizes the information from multiple coordinated investments in participants or communities.

If more CSBG funding must be diverted from crucial community programming, in order to design an effective, integrated reporting system, the major investments should begin by testing such a local system or data warehouse at the largest and most programmatically complex agencies. The results of this demonstration should inform the subsequent design of

National Community Action Foundation ROMA NG Final Comment the management and outcome measurement systems. Those, in turn, should be the basis for the reporting system and software adopted for collecting meaningful, valid state reports.

Thank you for taking the time for deep consideration of our major concerns. Please do not hesitate to contact me with any further questions.

Comment 84 (National Community Partnership ; Other/Unknown in ND)

Question 1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

1. Module 1: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported and creates a misuse of the Organizational Standards. Expecting constant perfection is unrealistic and does not have practical utility for local agencies or the State CSBG Lead Agency. As proposed, the single metric the State Agencies will report on will be the number of eligible entities that meet 100% of the CSBG Organizational Standards. To increase utility of the information, State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

2. Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency. This is not necessary for agency performance, and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will be skewed by large population centers and decrease the practical utility of the data even further. In addition, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems. In addition, as many local agencies serve families over time, with no definition of “new” being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years (e.g. Early Head Start, child care, after school programs, youth programs, job training, WIC, food boxes, LIHEAP, etc.).

3. Module 3: Remove the Community Outcome Indicators that include percent or rates. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

4. Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.

5. Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the eligible entity’s work. One option duplicates data already collected and the second uses a tool that defines financial stability based on a construct the uses assumptions drawn from the general population, not low-income individuals and families specifically. Because financial well-being means something very different for the low-income population in comparison to the general population, use of the scale will produce data of limited utility.

6. Address the unique issues of Public entities that receive CSBG. Public agencies (sub-grantees) that sub-contract most or all of their CSBG funding to other service providers may face added challenges of requiring their grantees to collect data in accordance with ROMA Next Generation requirements since it may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet the reporting requirements. This challenge is not addressed in the proposed reporting package.

7. Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network for ongoing training, technical assistance, and general capacity-building supports.

Question 2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.
The Partnership urges local agencies and states to estimate their time and cost burden. Based on feedback from the Network, we suspect that the current estimation is low given the increase in the amount of information required and the challenges many agencies face collecting data from multiple reporting systems. Agencies that respond to the estimation of burden question should choose one of the following options and include an explanation detailing how they arrived at the number and the impact of this collection on the agency or the state office. (Please forward your estimate to NCAF and the Partnership to assist in the estimated burden Network-wide.)

· Concerns include:

1. The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

2. The burden on local agencies is not impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems.

3. Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data.

In addition, it is not clear how the burden for local agencies is decreased by the online data collection system described in the by OCS. OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden outlined here.
Question 3: The quality, utility, and clarity of the information to be collected.

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

· Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

· Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

· Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

· As noted above, OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

· OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 85 (Nek-Cap, Inc. (Northeast Kansas Community Action Agency); Eligible Entity/Local Agency in KS)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility:

I support the collection of data outlined in each Module and believe it is necessary in informing Congress and all funders the results, outcomes and impacts of Community Action services and strategies. The baseline data elements are already collected and will present no additional burden. Specifically tracking and determining which specific services actually produce the final expected outcomes are more difficult at both the family and community level. To determine long term impact based on interaction with a CSBG funded service is currently not possible through existing software and would be very difficult and costly to measure without significant consumer self -reporting and community alignment with agency initiatives.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information:

The estimated cost to NEK-CAP, Inc., for the full implementation of ROMA Next Generation and proposed CSBG Annual Report is $264,220.00. This amount reflects approximately one-half of all CSBG funds currently received by the agency and would , therefore, need to supplemented with at a minimum an additional $76,000.00 In order to properly collect, analyze and evaluate data as required per the ROMA cycle. The estimate concluded that at a minimum approximately 8,880 hours of staff time would need to be devoted to the overall process. Furthermore, the agency would need to hire additional staff, restructure the job duties and services provided by currently CSBG funded staff and acquire new software modules either through CAP 60 or another software vendor with the capability of integrating all agency reported data. These estimates are based on current and projected costs associated with agency CAP 60 user’s time and wages, staffing patterns, training needs, software costs, and the potential for contracted services.

3. The quality, utility, and clarity of the information to be collected:

The quality of the data is only as good as the staff input and the accuracy and reporting capability of the data software system. CAP 60 does not produce quality reports and there are significant questions about validity and reliability. The information to be collected is quite clear but the utility of collecting such data has the potential to fail to demonstrate the overall impact of the utilization of CSBG funded strategies due to the inability to track long term outcomes and the systemic challenges, endured by low-income populations and communities, which are beyond the control of Community Action Programs.
4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

A software system which can integrate/extrapolate data from multiple systems would be ideal. Duplicate entry into various systems is a waste of resources and can affect the unduplicated count. A state supported system such as HMIS could potentially be adapted to meet the reporting requirements; however, there would more than likely be a significant cost with the redesign of such a system. Develop a mobile application which interfaces with agency tracking software that can be utilized by consumers for reporting purposes.

Comment 86 (New England Community Action Partnership (Necap) ; State Association in CT)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

There are numerous areas where the collection of information does not meet the threshold of necessity or utility. The package as proposed is burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided below that outlines where changes need to be made to reach a level of workability and utility.

· Module 1: Modify the options CSBG State Lead Agencies/grantees have when reporting to

OCS on CSBG Organizational Standards to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported and creates a misuse of the Organizational Standards. Expecting constant perfection is unrealistic and does not have practical utility for local agencies or the State CSBG Lead Agency. As proposed, the single metric the State Agencies will report on will be the number of eligible entities that meet 100% of the CSBG Organizational Standards. To increase utility of the information, State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

· Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency. This is not necessary for agency performance, and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will be skewed by large population centers and decrease the practical utility of the data even further. In addition, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems. In addition, as many local agencies serve families over time, with no definition of “new” being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years (e.g. Early Head Start, child care, after school programs, youth programs, job training, WIC, food boxes, LIHEAP, etc.).

· Module 3: Remove the Community Outcome Indicators that include percent or rates. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or Community Action Partnership ROMA Next Generation Talking Points Memo 4 targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

· Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.

· Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the eligible entity’s work. One option duplicates data already collected and the second uses a tool that defines financial stability based on a construct the uses assumptions drawn from the general population, not low-income individuals and families specifically. Because financial wellbeing means something very different for the low-income population in comparison to the general population, use of the scale will produce data of limited utility.

· Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US
counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network for ongoing training, technical assistance, and general capacity-building supports

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information.

Many agencies have expressed concerns about the cost and time of modifying and managing current systems comply with the draft CSBG Annual Report. NECAP echoes these concerns. Estimated costs will be reflected in agency-specific comments.

3. The quality, utility, and clarity of the information to be collected.

Several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level. It also mandates the use of limited outcome indicators in order to standardize and collapse the rich variety of Community Action Agency achievements and projects into a national summary.

· Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

· Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

· Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

· The draft report nearly triples the amount of now-required information. There are more than 1,100 items for local reports (and even more when counting "other" items) versus the over 350 in the current system. This means CAAs will be forced to contribute on many of the new state data items. Agencies will have to contribute because States must now investigate and report narrative details on local agency achievement of organizational standards. While OCS’s Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload),” it is important to note these concerns. Also, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

· OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 87 (New York Department Of Youth And Community Development ; State in NY)

It is unclear how HHS would make use of the distinction between “new” individuals/households among “total” individuals/households, served each year.

2. The number of data elements is overly burdensome, and it is not clear how useful this data will be in the interest of aggregating the impact of the CSBG network

3. Public and larger Community Action Agencies would now be required to collect significantly more data from their sub-recipients, without any additional funding being made available to assist with data collection. While the opportunity to enter data directly into HHS’ system once collected seems promising, individual data systems will need to be created and programmed to obtain the data being requested (there appear to be about three times as many data elements proposed as are in the current Information Survey)

4. The Information Survey should continue to inquire about Agencies’ own efforts to increase their capacity to achieve results. It is important to continue to illustrate the needs for ongoing training and technical assistance and Agencies’ efforts to meet such demands.

Comment 88 (New York State Community Action Association; Eligible Entity/Local Agency in NY)

Question 1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

The proposed collection of information includes some useful elements but in whole is quite burdensome and includes information that is of limited use and well beyond the parameters of the Community Services Block Grant (CSBG.) Suggested modifications which could result in a useful and effective system of information collection are noted below.

1. Module 1: Allow CSBG State Lead Agencies/grantees to report on the eligible entities’ compliance with standards at a range of levels beyond 100%, for example, 50%, 75% 90% and 100%. The Organizational Standards were intended to promote excellence not merely the lowest level of compliance. It is therefore reasonable and important that the metrics include those agencies that meet less than 100% to provide a more accurate assessment of the training and technical assistance needs of eligible entities and to track CAA’s progress in meeting the Standards over time. Requiring that only 100% compliance be reported undermines the intention of the standards to promote excellence and is of limited to utility for the CAAs, State lead agencies and NASCSP and OCS.

2. Module 3: Make the reporting of community level data on social indicators optional and provide flexibility in defining which indicators to use and report. The inclusion of Community Outcome data with percentages or rates provides little meaningful data on the initiatives undertaken by local agencies. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives.

3. Module 3: Collective Impact is only one of a variety of options for community-level work and should therefore be removed. As one in a menu of options, collective Impact should be removed as including it suggests a strong preference for this approach as opposed to other possible community-level strategies.

4. Module 4: Remove the Characteristics for NEW Individuals and Households Report. There is no definition of “New” included here nor is there a widely accepted standard for identifying a “new” customer. For example when is a customer who receives multiple services over many years “New”? And what utility does collecting this data have for state agencies or OCS when it is rolled up and aggregated? The purpose identified by OCS for the collection of this data is to inform the needs assessment and program development process. Agencies already have a process for identifying needs and developing services with little to be gained by expending the time and resources needed to collect this data. Furthermore, many agencies lack the technical capacity and resources to collect and analyze this data and many are challenged by the barriers created by the mandated use of multiple, often incompatible, reporting systems.

5. Module 4: The Stability Indicators included here are neither practical nor useful and should be removed. One option included here uses a tool for defining financial stability that is based on assumptions drawn from the general population and is therefore of very limited utility when applied to the low income people served by CAAs. Another option duplicates data already collected.

6. Reinstate the ROMA Goal: Agencies Increase their capacity to achieve results. This goal reflects the vital role of the national network of local CAAs and there is little sound reason to remove it. Agency capacity is critical to the success of the services provided and should be identified as such in the National Goals. Inclusion of Agency Capacity is also important to highlight the ongoing need for training, technical assistance and other capacity building services for the CAA network.

Question 2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.

While this is a question that can only be answered by individual CAAs, this Association is concerned about the burden of this enormous increase in the collection of information with no additional resources provided to meet this requirement. The increase in the data elements included in the proposed CSBG Annual Report from 300 to more than 1,000 poses potentially great burdens with no additional resources. This is particularly problematic given the challenges already faced by agencies that are required to collect and report data using multiple, often incompatible data systems.

It should also be noted that the on-line system described by OCS in the Dear Colleague Letter of June 17, 2016, may be of benefit to State Lead Agencies but does little to address the issues that eligible entities face in tracking and reporting data for multiple agencies using multiple information and reporting systems.

Question 3: The quality, utility, and clarity of the information to be collected.

A number of the elements to be collected are primarily useful as components of community-wide scorecards and have very limited utility or are not available for single agency reporting or as data to describe the work done in the smaller geographic areas served by local agencies, particularly regarding community-level work. The use of such data to assess and inform individual agency outcomes is at best limited and may in fact be misleading if used to compare agency outcomes with community indicators.

· Module 3, Section A: The indicators used here are to report community-level data to States and by States to OCS and will be of limited utility. The available community-level data often do not match the geographic area and population targeted by the agency, for example by census track or age. It is also extremely difficult to assess the impact of an agency’s intervention as part of a community-level initiative since many external factors influence the impact of such an undertaking and could erase or minimize the impact of community-level initiatives. Evaluating the outcomes of community-level initiatives requires a formal program evaluation and cannot be accomplished solely through the use of data collected in the Annual Report.

· Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form should be removed. As noted above the collection of this information is both difficult and costly and of little use at the local level. This data does not include information about the needs and challenges facing “new” customers and does little to help agencies assess whether they are meeting their needs. Additionally, this data fails to account for the many “one time” customers who access services such as LIHEAP and thus makes interpretation of the data difficult and inaccurate. Further, as previously noted, because many agencies are required to use multiple, incompatible data collection and reporting systems, this will pose a significant burden on agencies with no resources available to support this work.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

As previously noted, the on-line system referenced by OCS in the June 17, 2016 Dear Colleague Letter will do little to help agencies track customers, services and outcomes. The online automated system may be an asset to State Lead Agencies, but offers little for local Community Action Agencies which will still be required to track and report data using multiple information management systems with different, often incompatible requirements. It is unlikely that the State Lead Agencies will have sufficient resources to enhance the network’s data collection capacity nor will the proposed on-line system address the need for improved information management capacity as required by ROMA Next Gen. To minimize the burden and maximize the utility of the data collected, the federal government should assess its data collection requirements and find a way to use information management systems that can be integrated to meet the needs of the multiple government funding and regulatory agencies.

Comment 89 (Ninth District Opportunity, Inc.; Eligible Entity/Local Agency in GA)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

3. The quality, utility, and clarity of the information to be collected; and

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Comment 90 (North Central Community Action Program; Eligible Entity/Local Agency in WI)

Thank you for your work on ROMA Next Generation. I appreciate the overwhelming task this is to develop a system that captures the great work that is performed across the country by community action agencies as well as demonstrate quality service and data collection.

First of all, I am concerned about the burden of reporting and tracking of all of the data. We are a small community action agency. Our organizational structure will make my job much more complicated because I do much of the reporting. The requirements in the proposed ROMA NG will be extremely challenging for me. I did like some of the changes that came with the first revisions to ROMA NG but I still believe that you are asking for too much information, some of which is not very practical or useful. We do not have the financial resources to put into place all of the required data collection components, let along all of the time needed, to address everything in ROMA NG.

A few areas that stand out as unrealistic include:

-Collecting over 1,000 data elements in the proposed IS Survey. The current survey is challenging and only has a little over 300.

-The data requested will require us to implement new data collection methods that are cost prohibitive.

-The time necessary to respond will go way beyond the estimated hour’s projections.

Comment 91 (North Dakota Community Action ; State Association in ND)

While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

I. Module 1: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported and creates a misuse of the Organizational Standards. Expecting constant perfection is unrealistic and does not have practical utility for local agencies or the State CSBG Lead Agency. As proposed, the single metric the State Agencies will report on will be the number of eligible
entities that meet 100% of the CSBG Organizational Standards. To increase utility of the information, State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs.

For instance State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the
collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local
agency. This is not necessary for agency performance, and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will be skewed by large population centers and decrease the practical utility of the data even further. In addition, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems. In addition, as many local agencies serve families over time, with no definition of "new" being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes
challenging when they may receive different services over many years (e.g. Early Head Start, child care, after school programs, youth programs, job training, WIC, food boxes, LIHEAP, etc.). Module 3: Remove the Community Outcome Indicators that include percent or rates. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.

Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the
eligible entity's work. One option duplicates data already collected and the second uses a tool that defines financial stability based on a construct the uses assumptions drawn from the general population, not low-income individuals and families specifically. Because financial well-being means something very different for the low-income population in comparison to the general population, use of the scale will produce data of limited utility.

Address the unique issues of Public entities that receive CSBG. Public agencies (sub-grantees) that sub-contract most or all of their CSBG funding to other service providers may face added challenges of requiring their grantees to collect data in accordance with ROMA Next Generation requirements since it may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet the reporting requirements. This challenge is not addressed in the proposed reporting package.

Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US
counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network for ongoing training, technical assistance, and general capacity-building supports.

2. The accuracy of the agency's estimate of the burden of the proposed collection of information Regarding the accuracy of the agency estimate of the cost burden, CAPND concerns include:

The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

The burden on local agencies is not impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems. An initial investment in a new software for North Dakota was quoted at over $50,000 which only covers the set-up and one year of training. This does include any of the costs association with adding necessary modules such as Weatherization or Head Start.

Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data.

In addition, it is not clear how the burden for local agencies is decreased by the online data collection system described in the by OCS. OCS' Dear Colleague Letter of June 17, 2016, states,

"Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and — at the discretion
of the states — the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)." It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden outlined here.

3.The quality, utility, and clarity of the information to be collected

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value,
meaning, and usefulness at the local level.

• Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of "one time" customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

• As noted above, OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and — at the discretion of the states — the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)."

It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and "talk" to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 92 (North Dakota Department Of Commerce; State in ND)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

· While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG).

· The burden on local agencies is not impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

· As a State grantee, we do not have the funding or capacity for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden or tripling the amount of data to be collected.

· In regards to the changes to Module 4, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems.

· The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

· Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data.

3. The quality, utility, and clarity of the information to be collected; and

· Regarding the stability indicators selected and included in the clearance package (Module 4), they are not helpful or practical when reporting customers stabilized as a result of the eligible entity’s work. Tracking clients after they have left the program would be overly cumbersome and many individuals may be difficult to contact. This has the potential to consume a large amount of agency resources.

· Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

· Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency.

· OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

Comment 93 (North Dakota Lead Agency; State in ND)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

· While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG).

· The burden on local agencies is not impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information


· As a State grantee, we do not have the funding or capacity for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden or tripling the amount of data to be collected.

· In regards to the changes to Module 4, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems.

· The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

· Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data.

3. The quality, utility, and clarity of the information to be collected; and

· Regarding the stability indicators selected and included in the clearance package (Module 4), they are not helpful or practical when reporting customers stabilized as a result of the eligible entity’s work. Tracking clients after they have left the program would be overly cumbersome and many individuals may be difficult to contact. This has the potential to consume a large amount of agency resources.

· Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

· Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency.

· OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

Comment 94 (Northeast Kansas Community Action Agency; Eligible Entity/Local Agency in KS)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility:

I support the collection of data outlined in each Module and believe it is necessary in informing Congress and all funders the results, outcomes and impacts of Community Action services and strategies. The baseline data elements are already collected and will present no additional burden. Specifically tracking and determining which specific services actually produce the final expected outcomes are more difficult at both the family and community level. To determine long term impact based on interaction with a CSBG funded service is currently not possible through existing software and would be very difficult and costly to measure without significant consumer self -reporting and community alignment with agency initiatives.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information:

The estimated cost to NEK-CAP, Inc., for the full implementation of ROMA Next Generation and proposed CSBG Annual Report is $264,220.00. This amount reflects approximately one-half of all CSBG funds currently received by the agency and would , therefore, need to supplemented with at a minimum an additional $76,000.00 In order to properly collect, analyze and evaluate data as required per the ROMA cycle. The estimate concluded that at a minimum approximately 8,880 hours of staff time would need to be devoted to the overall process. Furthermore, the agency would need to hire additional staff, restructure the job duties and services provided by currently CSBG funded staff and acquire new software modules either through CAP 60 or another software vendor with the capability of integrating all agency reported data. These estimates are based on current and projected costs associated with agency CAP 60 user’s time and wages, staffing patterns, training needs, software costs, and the potential for contracted services

3. The quality, utility, and clarity of the information to be collected:

The quality of the data is only as good as the staff input and the accuracy and reporting capability of the data software system. CAP 60 does not produce quality reports and there are significant questions about validity and reliability. The information to be collected is quite clear but the utility of collecting such data has the potential to fail to demonstrate the overall impact of the utilization of CSBG funded strategies due to the inability to track long term outcomes and the systemic challenges, endured by low-income populations and communities, which are beyond the control of Community Action Programs.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

A software system which can integrate/extrapolate data from multiple systems would be ideal. Duplicate entry into various systems is a waste of resources and can affect the unduplicated count. A state supported system such as HMIS could potentially be adapted to meet the reporting requirements; however, there would more than likely be a significant cost with the redesign of such a system. Develop a mobile application which interfaces with agency tracking software that can be utilized by consumers for reporting purposes.

Comment 95 (Northeast Pennsylvania Community Action Agency; Other/Unknown in PA)

The proposed collection of information is necessary as it pertains to those programs that are operated by each entity. The information collected can be used as a tool for perspective agencies to track areas where there is an increase or decrease in demand for services. It can also be used as a way to determine if entities should seek additional funding opportunities in particular areas. Although it is nearly impossible to provide an estimate on the true cost per agency, the Health and Human Services average burden hours seems to underestimate this massive undertaking. We have included with these comments, a starting point budget
justification, to compare to Health and Human Services. This identifies just how drastically different a Sub-Grantees burden hours would be in relation to the average figures calculated by HHS. Not taken into consideration are varied costs and time associated with software development which includes development of reporting tool'.

The burden on the agencies data collection will be substantial. The process will first begin with the computer/client tracking software that the agency currently uses. It will have to be determined if this mechanism will meet the new data collection standards and if the software is capable of handling the increased quantity of information. The software will not only have to be updated for the end users but also new reporting documents will have to be developed. If the current tracking mechanism is found unsuitable, then the appropriate staff will have to begin researching and reviewing suitable alternatives. If any changes are made to the software and/or reporting tools, this will require retraining both end users and those staff responsible for analyzing reports. With the increase of reportable areas, this will increase the amount of time those responsible for aggregating this data will spend on this task. Some areas may also require manual tracking if they cannot be integrated into a new or existing software system.

The information collected is only as good as the corresponding questions that are asked. The questions need to be clear and concise. They should not be phrased in ways that may lead to misinterpretation or open tenderness. If any agency is not currently using a form of information technology, it will be near impossible to collect, analyze and report on all the data to be collected. Whether a current software system is in place or not, there will be an increase to each entity that will be charged with undertaking this process.

We appreciate the opportunity to express our views on the Community Services Block Grant (CSBG) Annual report. We are hopeful the information contained in these comments will allow the Office of Community Services to obtain insight on the daily processes of those who will be responsible for administering, collecting and reporting on this vital program.
Community Services Block Grant (CSBG) ROMA Next Generation Implementation Cost Estimate

OPERATING COSTS:

Full Cost of the different levels of staff contribution time (approx. 30 staff x 5%Of time)

Contracted Professional services and training (approx. I staff x 10% of time)

Maintaining software, integrating mandated data (approx. 2 staff x 5% of time)

Tracking projects (approx. 2 staff x 5% of time)

Reporting progress on organizational standards (approx. I staff x 5% of time)

Support and train staff who use software (approx. 1 staff x 10% of time)

ROMA training ( approx. 1 staff x 5 % of time)

External Data Acquisition ( approx. 15 staff x 5% of time)

Total Operating

GRAND TOTAL

Rate

$15.00 $15.00

$15.00

$15.00
$15.00

$15.00

$15.00

$15.00

$15.00

Hours on Project

2925

195

195 195

97.5 195
97.5

1462.5

53625

$43,875 $2,925

$2,925

$2,925

$1 ,463 $2,925

$1463

$21 ,938

$80,438

$80,438

Comment 96 (Northern Tier Community Action Corp. ; Eligible Entity/Local Agency in PA)

Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

There are individual parts of this effort that will be of benefit in reporting, however when you offset this with the additional burden of data collection and reporting, it is difficult to justify.

Operational costs increase continually, with no increase in CSBG funding. CSBG is the core funding for this agency to leverage and facilitate a variety of programs that could not stand alone. These critical programs and services for low-income families are in many cases the only thing that keeps families from being, unemployed, hungry and homeless. As it stands now our staff typically manages several programs at one time and in the process of managing these programs, serve as case managers helping families to hold it all together. Now we are to consider adding an even greater burden with tracking additional elements and reducing the time they have to do the work of service provision and case management.

The accuracy of the agency estimate of the burden of the proposed collection of information

It is understood that the time estimated for reporting would decrease to some extent as the process is perfected, however it will still be an unbearable burden on the minimally funded and staffed agencies. This would require additional staff just to provide data collection and reporting. We have calculated the burden of proposed collection based on the current time spent collecting and compiling data for reports and then completing the reports. Northern Tier Community Action Corporation is an Equal Opportunity Employer and Provider of Services Programs and services made possible with funding by the Community Services Block Grant through the Department of Community and Economic Development

The current reporting for the Pennsylvania quarterly Community Organization Planning & Outcomes System (COPOS) reports and the Organizational Standards reporting we provide to DCED requires an average of 237 hours annually. The burden of increasing this threefold (711hrs. and possibly over $20,000 annually) for agencies such as ours that are minimally staffed and underfunded is overwhelming; and to say the least cost prohibitive.

The quality, utility, and clarity of the information to be collected; and Data for some of the indicators in the report is not available in many of our areas and will require establishing additional arrangements with other organizations that are also overburdened, understaffed and underfunded, trying to do the best they can with what they have to serve the needs of our communities. Then if you look at the level of data collected and reported in many cases is of little value in our planning and evaluation efforts.

Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

It has always seemed a simple solution to reducing the reporting burden on local agencies would be for OCS to create a national and statewide client/program tracking system where every client/ program would be entered into the system with the indicator and outcomes data collected and entered during the intake, service provision and outcome process. Once data is entered into the system, OCS and State Agencies could pull reports on a continual basis. We understand the massive scope of such a project as creating this system, but once implemented, it would free up local agency staff to do the important job of serving the people and not spending an inordinate amount of their time collecting data and preparing reports.

Typically each program funding agency has requirements for reporting each of their programs (i.e. Homeless Prevention Programs, Foreclosure Prevention Programs require reporting to HUD, Energy and Weatherization Programs are required to report to DOE, etc.) We have a wealth of human capital that truly care about taking care of the needs of low-income families and helping them to achieve self-sufficiency, but much of their time and abilities is lost on data collection and reporting.

Comment 97 (Northwest New Jersey Community Action ; Eligible Entity/Local Agency in NJ)

COST FACTORS

I would like to focus my comments on the financial burden that I believe the system will cost. The BIGGEST issue is that funding sources like their own proprietary software and won’t share information or allow us to import export data to our central database. As you can imagine, with minimal funding for our programs, we don’t have the staff time to double enter all the data. (Currently, these federal funding sources will not cooperate: WIC, Weatherization, LIHEAP Energy Assistance, Child Care Block Grant, Head Start, Early Head Start and HUD.) Additionally, there are many State programs that will not do it either. The irony is, in our own state (NJ) our CSBG cognizant agency (NJ Department of Community Affairs) requires us to have a central database, and they run some of these programs (LIHEAP, WX HUD), but they won’t allow the data to be exported!

So what if we were forced to enter all this information, what would it cost? Here’s my conservative estimate:

· It now takes about a half hour to do a family. There are 300 factors going to 1000 (as I understand it), but let’s just say it’s now an hour for round terms.

· We don’t know how many unduplicated clients we have because of the funding source software issue previously mentioned. We believe that there are about 35,000 clients, but let’s low ball it at 20,000; so that’s 20,000 hours.

· Assume data clerks make (with payroll taxes and fringe) $50,000 a year.

· Assume they work 40 hours a week.

· 20,000 hours represent 500 weeks.

· That’s 10 data clerks x $50,000 = $500,000.

· Add in the cost of the software, about $5,000 per year annual fee.

· Add in an internal quality management/analysis person at $105,000 (includes health and fringe)

· TOTAL = $610,000

· I would ask:

o Is it worth the cost?

O I’d rather put that money into services.

O Who is going to pay it?

O Will I have to pull program staff out and not get the services done at the expense of reporting?

I appreciate what you are trying to do and on a theoretical basis, I endorse it, BUT until the practical side of this is addressed, it will render us ineffective because we will be focused on reporting and not service. Community action absolutely needs to tell a better story and it has been one of our weaknesses. We are complex organizations making profound changes to our communities and we always walk a tightrope trying to do well on reduced budgets. I’m with you in the “data collection spirit,” but not on practical implementation side.

Comment 98 (Norwescap; Eligible Entity/Local Agency in NJ)

Thank you for the opportunity to comment on the ROMA Next Generation concepts. While there are a number of items in ROMA NG that don’t have much value, I know that you are receiving comments from the National CAP Association that mirror my feelings on them. So please consider my unwritten comments a “ditto” of theirs when it comes to the burden of collecting information that may be difficult to get and at the same time have no value to the overall management and direction of the agency.

COST FACTORS

I would like to focus my comments on the financial burden that I believe the system will cost. The BIGGEST issue is that funding sources like their own proprietary software and won’t share information or allow us to import export data to our central database. As you can imagine, with minimal funding for our programs, we don’t have the staff time to double enter all the data. (Currently, these federal funding sources will not cooperate: WIC, Weatherization, LIHEAP Energy Assistance, Child Care Block Grant, Head Start, Early Head Start and HUD.) Additionally, there are many State programs that will not do it either. The irony is, in our own state (NJ) our CSBG cognizant agency (NJ Department of Community Affairs) requires us to have a central database, and they run some of these programs (LIHEAP, WX HUD), but they won’t allow the data to be exported!

So what if we were forced to enter all this information, what would it cost? Here’s my conservative estimate:

· It now takes about a half hour to do a family. There are 300 factors going to 1000 (as I understand it), but let’s just say it’s now an hour for round terms.
· We don’t know how many unduplicated clients we have because of the funding source software issue previously mentioned. We believe that there are about 35,000 clients, but let’s low ball it at 20,000; so that’s 20,000 hours.

· Assume data clerks make (with payroll taxes and fringe) $50,000 a year.

· Assume they work 40 hours a week.

· 20,000 hours represent 500 weeks.

· That’s 10 data clerks x $50,000 = $500,000.

· Add in the cost of the software, about $5,000 per year annual fee.

· Add in an internal quality management/analysis person at $105,000 (includes health and fringe)

· TOTAL = $610,000

· I would ask:

o Is it worth the cost?

O I’d rather put that money into services.

O Who is going to pay it?

O Will I have to pull program staff out and not get the services done at the expense of reporting?

I appreciate what you are trying to do and on a theoretical basis, I endorse it, BUT until the practical side of this is addressed, it will render us ineffective because we will be focused on reporting and not service. Community action absolutely needs to tell a better story and it has been one of our weaknesses. We are complex organizations making profound changes to our communities and we always walk a tightrope trying to do well on reduced budgets. I’m with you in the “data collection spirit,” but not on practical implementation side.

If you have questions about this, please contact me at the number below and PLEASE be realistic about your expectations. Thanks.

Comment 99 (Northeast Community Action Agency ; Other/Unknown in FL)

No Comment

Comment 100 (Nothing Compares; Other/Unknown in NC)

As indicated in our initial response to the proposed annual report, Nonh Carolina fully understands the need for and embraces the National Performance Management Framework for administering the Community Services Block Grant (CSBG). In addition,

NC applauds the efforts and expertise of the National Association for State Community Services Programs (NASCSP) staff who has and continue to work to ensure that ROMA Next Gen meets the expectations of the Office of Community Services while best representing the great work of the CSBG Network. However, we remain concerned that the collection of information as proposed is not necessary for proper performance planning and analysis, will cause undue burden on both state and local human and
financial resources, lacks clarity necessary to effectively determine community level work and imposes an aggressively unrealistic timeframe for implementation given the need for extensive training and technical assistance at all levels of the CSBG administration. The following feedback is offered relative to the specific modules of the proposed Annual Report:

Comment 101 (Ohio Association Of Community Action Agencies; Other/Unknown in OH)

Ohio’s fifty community action agencies have reviewed the new ROMA Next Generation Modules and determined that the cost would be, on average, equal to anywhere from 10% to 16% of their total CSBG award. Of those sent to me, most were closer to the 16 % range. This is burdensome, to say the least, and not as Congress intends the money to be spent.

We are also concerned about the requirements in the various modules and rather than write to all of the points, I think it would be more efficient to tell you that we agree with the comments of the Community Action Partnership. The only exception would be the points made about public agencies, as Ohio has only private, nonprofit Community Action Agencies.

Comment 102 (Opportunities For Otsego, Inc. ; Other/Unknown in NY)

I would like to thank and commend you for the work that has been done regarding ROMA Next Generation. Opportunities for Otsego participated in much of the discussion and review that took place at the state and national levels. This agency's responses are in line with what my colleagues concluded after careful consideration of ROMA Next Generation. Recommendations are as follows:

Comment 103 (Oregon Housing And Community Services ; Other/Unknown in OR)

Thank you for providing an opportunity to provide feedback on ROMA Next Generation data collection and the proposed changes to the CSBG Annual Report.

In regards to the four questions put forth in the Federal Register, we support the feedback given by representatives of our regional network of community action agencies in Oregon, Washington, California and Idaho. All through this process, valid points of concern have been raised by the network regarding the effectiveness of the process, the usefulness of the data, and the capacity to collect, analyze and report the data. As they are the agencies whom juggle the burden of increased data collection and reporting, with increasingly insufficient tools, we support their expertise on the subject.

We also would like to underline their concerns about capacity. This is the message that we hear often from our agencies: they simply do not have the staffing and financial resources to continue to increase their data collection without the introduction of supporting tools to do so, or without a clear picture as to the effectiveness and usefulness of the data.

Oregon’s homeless and anti-poverty delivery system is comprised of eighteen community action agencies. This network is a combination of large and small public agencies and nonprofits residing in and serving both urban and rural populations. Despite their differences in service areas and capacity, a common thread is the increasing burden of data collection and the difficulties to collect accurate data for multiple reports using multiple required databases. Agencies spend an inordinate amount of staff time retrieving data, and then attempting to reconcile the data and unduplicated it, as the databases often do not communicate with each other or work together. As these demands increase across funding sources, less staff time is devoted to providing services. Agencies have already expressed that they do not have the capacity to report data with the anticipated increased level of complexity. While we appreciate the work put forth on these projects, we must support the agencies’ concerns that these new additional measures and how they will be implemented be given greater time and consideration.

Comment 104 (People Incorporated Of Virginia; Other/Unknown in VA)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

While the proposed Annual Report contains some elements that will be useful, particularly at the Individual and Family levels, there are a number of areas and indicators that cause the Board and staff of People Incorporated grave concern. Specific areas are detailed below, and focus largely on the Community level indicators. In its current iteration, the proposed CSBG Annual Report is overly burdensome, does not provide for consistent data across agencies and states, and requests extensive information outside the mandate and scope of the Community Services Block Grant. Much of the information requested is inappropriate and has limited, if any practical utility.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

The agency’s’ estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under what we estimated. Based on staff calculations, the burden to People Incorporated would be 2,652 hours in the year before reporting begins, 3,094 hours in the first year of reporting and 2,522 hours per year thereafter. The cost of this time burden is estimated to be $82,812, $74,138 and $60,622 in years one, two and thereafter. This accounts only for staff time; the new IT system necessary to track and report the new data elements would likely double this cost.

The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

3. The quality, utility, and clarity of the information to be collected; and

People Incorporated's greatest concern in regard to this proposed report are in the area of the Community level indicators. The social or population level indicators being requested in Module 3 are inappropriate for individual agency reporting and are best utilized for community-wide scorecards or as part of a community needs assessment. Comparing overall percentages of community indicators to individual local agencies outcomes is misleading. As an example, this agency does extensive work in economic development, including job creation, but in the last year, the gains staff have been able to make in job creation in SW Virginia are more than offset by mine and plant closures, making reporting on the rate of unemployment meaningless in terms of our agency performance; the hundreds of jobs People Incorporated has created are subsumed in the larger number of layoffs when utilizing the unemployment rate as an indicator. This is true for many, if not all of the proposed social indicators,
which include, but are not limited to the change in communities' rates of infant mortality, childhood obesity, adult literacy, recidivism, high school graduation, homeownership, foreclosure, et cetera.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” This online system and any new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

The online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and often separate data collection systems. People Incorporated is required by funders to use more than 10 discrete data systems, no two of which are designed to communicate readily with an outside system.

OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 105 (Proaction Of Steuben And Yates Inc.; Eligible Entity/Local Agency in NY)

Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

1. Module 1: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported and creates a misuse of the Organizational Standards. Expecting constant perfection is unrealistic and does not have practical utility for local agencies or the State CSBG Lead Agency. As proposed, the single metric the State Agencies will report on will be the number of eligible entities that meet 100% of the CSBG Organizational Standards. To increase utility of the information, State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

2. Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency. This is not necessary for agency performance, and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will be skewed by large population centers and decrease the practical utility of the data even further. In addition, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems. In addition, as many local agencies serve families over time, with no definition of “new” being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years (e.g. Early Head Start, child care, after school programs, youth programs, job training, WIC, food boxes, LIHEAP, etc.).

3. Module 3: Remove the Community Outcome Indicators that include percent or rates. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

4. Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.

5. Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the eligible entity’s work. One option duplicates data already collected and the second uses a tool that defines financial stability based on a construct the uses assumptions drawn from the general population, not low-income individuals and families specifically. Because financial well-being means something very different for the low-income population in comparison to the general population, use of the scale will produce data of limited utility.

6. Address the unique issues of Public entities that receive CSBG. Public agencies (sub-grantees) that sub-contract most or all of their CSBG funding to other service providers may face added challenges of requiring their grantees to collect data in accordance with ROMA Next Generation requirements since it may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet the reporting requirements. This challenge is not addressed in the proposed reporting package.

7. Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network for ongoing training, technical assistance, and general capacity-building supports. Question 2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.

The Partnership urges local agencies and states to estimate their time and cost burden. Based on feedback from the Network, we suspect that the current estimation is low given the increase in the amount of information required and the challenges many agencies face collecting data from multiple reporting systems. Agencies that respond to the estimation of burden question should choose one of the following options and include an explanation detailing how they arrived at the number and the impact of this collection on the agency or the state office. (Please forward your estimate to NCAF and the Partnership to assist in the estimated burden Network-wide.)

· Concerns include:

1. The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

2. The burden on local agencies is not impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems.

3. Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data.

· The agency’s estimate time burden of 242 hours per sub-grantee is significantly underestimated. Based on estimated calculations, the burden to our local agency will be more than 2,000 hours. The cost burden for the first year alone is estimated to be around $32,000.

· In addition, it is not clear how the burden for local agencies is decreased by the online data collection system described in the by OCS. OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden outlined here.

Question 3: The quality, utility, and clarity of the information to be collected.

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level. Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

· Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

· Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

· As noted above, OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 107 (Pueblo County Housing And Human Services; Other/Unknown in CO)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

I do not believe the ROMA CSBG report has value at all levels and is doable given our limited financial resources and technological capacity. This report places an undue burden on our agency and does not improve our performance management system. The package as proposed in its entirety is overly burdensome and contains reporting tools that will not provide useful information.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

I am not able to conduct a good-faith estimate of the time burden to implement ROMA Next Generation, I can assure you it could be more than the amount listed by the HHS table estimating the hours needed, on average burden of hours per response (164 hours). I believe this estimate may be low given the increase in the amount of information required and the challenges many agencies face collecting data from multiple reporting systems. The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources. The burden on our agency is not impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report our agency will need to modify existing systems. Our agency lacks a management information system with the capacity to provide the required data. In addition, our agency uses multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data. In addition, it is not clear how the burden for our local agency is decreased by the online data collection system described by OCS. It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit our local agency. We will still need a system that tracks customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems.


3. The quality, utility, and clarity of the information to be collected; and

Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading.

A characteristic for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at our local level. Since demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of “one time” customers in our local programs will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer our agency uses multiple reporting systems and will also have the added burden of tracking new customers across all programs and services.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

To minimize the burden on respondents, I suggest DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency. This is not necessary for agency performance. Our agency lacks a management information system with the capacity to integrate the required data given our use of multiple (and sometimes incompatible) reporting systems. In addition, our agency serves families in different programs over time, with no definition of “new” being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years.

Comment 108 (Redwood Community Action Agency ; Eligible Entity/Local Agency in CA)

This response is provided by a CSBG eligible entity engaged in day-to-day operations serving clients through a variety of programs which are leveraged extensively with other funding sources (and reporting requirements). We appreciate the opportunity to provide feedback and comments that may assist in developing the most appropriate annual report to capture the impact of CSBG programs.

Comment 109 (Sa Howell, LLC ; Other/Unknown in GA)

An agency level ROMA Goal needs to be included in order to track organizational capacity. Agency level activities are a fundamental principle of Community Action, and the inclusion of agency level outcomes are extremely necessary for tracking purposes.

Comment 110 (Semcac; Eligible Entity/Local Agency in MN)

Question 1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

1. Module 1: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported and creates a misuse of the Organizational Standards. Expecting constant perfection is unrealistic and does not have practical utility for local agencies. As proposed, the single metric the State Agencies will report on will be the number of eligible entities that meet 100% of the CSBG Organizational Standards. To increase utility of the information, State Agencies should have the option to report the

Dodge County

Fillmore County

Freeborn County

Houston County

Mower County

Steele County

Winona County

634-4350

765-2761

373-1329

725-3677

433-5889

451-7134

452-8396

With additional programs operated in Goodhue, Olmsted, Rice and Wabasha Counties

Please remember Semcac programs in your financial and estate planning. Your legacy is a gift to the future.

An Equal Opportunity Employer number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

2. Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local agency. This is not necessary for agency performance, and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will be skewed by large population centers and decrease the practical utility of the data even further. In addition, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems. In addition, as many local agencies serve families over time, with no definition of “new” being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years (e.g. Early Head Start, child care, after school programs, youth programs, job training, WIC, food boxes, LIHEAP, etc.).

3. Module 3: Remove the Community Outcome Indicators that include percent or rates. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

4. Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.

5. Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the eligible entity’s work. One option duplicates data already collected and the second uses a tool that defines financial stability based on a construct the uses assumptions drawn from the general population, not low-income individuals and families specifically. Because financial well-being means something very different for the low-income population in comparison to the general population, use of the scale will produce data of limited utility.

6. Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network for ongoing training, technical assistance, and general capacity-building supports.

Question 2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.

1. The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

2. The burden on local agencies is not impacted by the implementation of the Online Data Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems.

3. Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data.

4. The agency’s’ estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is significantly under estimated. Based on our calculations, the burden to our local agency would be 322 The cost of this time burden is estimated to be $6,440.00

· In addition, it is not clear how the burden for local agencies is decreased by the online data collection system described in the by OCS. OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees).

Question 3: The quality, utility, and clarity of the information to be collected.

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

· The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

· Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

· Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

· As noted above, OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and – at the discretion of the states – the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload).” It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

· OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 112 (Southeastern North Dakota ; Eligible Entity/Local Agency in ND)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

While there are elements that may be helpful, there are a number of areas detailed below where the collection of information does not meet the threshold of necessity or utility. The package as proposed in its entirety is overly burdensome, contains reporting tools that will provide useless information, and is outside the reach of the Community Services Block Grant (CSBG). However, with modifications, a system for collection of information can be developed that improves the performance management of the agency and has practical utility. Detail is provided here outlining where changes need to be made to reach a level of workability and utility.

1. Module I: Modify the options CSBG State Lead Agencies/grantees have when reporting to OCS on CSBG Organizational Standards to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported and creates a misuse of the Organizational Standards. Expecting constant perfection is unrealistic and does not have practical utility for local agencies or the State CSBG Lead Agency. As proposed, the single metric the State Agencies will report on will be the number of eligible entities that meet 100% of the CSBG Organizational Standards. To increase utility of the information, State Agencies should have the option to report the number of agencies at a variety of thresholds that still delineate strong performance or allow for a more accurate reflection of the training and technical assistance needs. For instance State Agencies should be able to report the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards. This will allow for the collection of more accurate information; better tracking of results over time; and better, more efficient investment of training and technical assistance.

Module 4: Remove the Characteristics for NEW Individuals and Households Report. This report has no practical utility at the local level given the time and expense to create it at each local
agency. This is not necessary for agency performance, and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will be skewed by large population centers and decrease the practical utility of the data even further. In addition, many agencies may either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and sometimes incompatible) reporting systems. In addition, as many local agencies serve families over time, with no definition of "nevi' being provided for review, the CSBG Network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years (e.g. Early Head Start, child care, after school programs, youth programs, job training, WIC, food boxes, LIHEAP, etc.).

Module 3: Remove the Community Outcome Indicators that include percent or rates. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives. Such data has validity for geographic areas much larger in than those addressed or targeted by local agency projects. This validity is lost when the scale is smaller and accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. It is more appropriate to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work and while CSBG eligible entities are engaged in such efforts, privileging one approach given the broader range of options for doing community-level work is inappropriate and unnecessary. This should be removed and included in training and technical assistance endeavors.

Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the
eligible entity's work. One option duplicates data already collected and the second uses a tool that defines financial stability based on a construct the uses assumptions drawn from the general population, not low-income individuals and families specifically. Because financial well-being means something very different for the low-income population in comparison to the general population, use of the scale will produce data of limited utility.

Address the unique issues of Public entities that receive CSBG. Public agencies (sub-grantees) that sub-contract most or all of their CSBG funding to other service providers may face added challenges of requiring their grantees to collect data in accordance with ROMA Next Generation requirements since it may limit agencies to contracting with providers that have the requisite data collection capacity or require additional technical assistance to meet the reporting requirements. This challenge is not addressed in the proposed reporting package.

Include a 4th ROMA Goal: Agencies Increase their capacity to achieve results. Reinstate this national goal that addressed the critical role the network of local agencies serving 99% of US counties plays in addressing poverty. Reasons for exclusion have been inadequate to date. Because agency capacity is critical to the ultimate success of programs and services, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the Network for ongoing training, technical assistance, and general capacity-building supports.

2. The accuracy of the agency's estimate of the burden of the proposed collection of information

Regarding the accuracy of the agency estimate of the cost burden, CAPND concerns include:

The current IS survey includes 300+ data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

The burden on local agencies is not impacted by the implementation of the Online Data

Collection system (OLDC). To comply with the draft CSBG Annual Report local agencies will need to modify existing systems. An initial investment in a new software for North Dakota was quoted at over $50,000 which only covers the set-up and one year of training. This does include any of the costs association with adding necessary modules such as Weatherization or Head Start.

Many agencies lack management information systems with the capacity to provide the required data. Data fields required for ROMA Next Generating reporting may not be mirrored in those used by agencies. In addition, many agencies use multiple (and sometimes incompatible) reporting systems which adds to the time necessary to provide the data.

In addition, it is not clear how the burden for local agencies is decreased by the online data collection system described in the by OCS. OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and — at the discretion of the states — the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual

It is important to note, this online automated system may entry and automatic upload)." benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden outlined here.

3.The quality, utility, and clarity of the information to be collected

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

Several of the elements to be collected as described are not available in smaller geographic areas usually targeted by local agencies in their community work and if created by local agencies, lose validity. The social or population level indicators being requested are not meant for single agency reporting and are best served for community-wide scorecards or utilized as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies outcomes is misleading

Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form is an unnecessary document that will not produce quality data; it will have little utility at a local level; and it lends itself to meaningless data at a state and national level. It should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of "one time" customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added burden of tracking new customers across all programs and services.

Module 3, Section A: Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility. Several challenges underscore this point. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

As noted above, OCS' Dear Colleague Letter of June 17, 2016, states, "Aware of the reporting burden the proposed Annual Report will entail, OCS will decrease that burden by establishing an on-line automated system for use by the states and — at the discretion of the states — the local agencies, for reporting that allows in many instances auto-populating data from one year to the next and a variety of data entry processes (e.g. manual entry and automatic upload)."

It is important to note, this online automated system may benefit State Lead Agencies (grantees), but will most likely not benefit local Community Action Agencies (sub-grantees). Sub-grantees (local agencies) will still need systems that track customers, services, outcomes, and given the requirement of many funding sources (including other federal Department of Health and Human Services funding) requires additional and sometime separate data collection systems. State grantees do not have the funding or capacity for the most part, to create systems that will streamline or decrease the burden outlined here. Furthermore, new state-level systems will do little to address the needs of local agencies for improved management information systems with the capacity to collect and report the data required by ROMA Next Generation.

OCS and the federal government provide funding to local sub-grantees that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and "talk" to one another to produce reports that meet the needs of local agencies, states, and OCS.

Comment 113 (Southern New Hampshire Services ; Eligible Entity/Local Agency in NH)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The proposed collection of information represents a significant increase from current practice, given that the proposed CSBG Annual Report essentially triples the data elements currently contained in the Information System Survey. We do not believe that all of the information is necessary, nor is it likely that a number of elements will have any practical usefulness, especially when aggregated across the 1,000 Community Action Agencies. Points spelled out below will identify more specifics in this area.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

We feel the time burden of 242 hours per agency is underestimated. Based on our calculations we estimate approximately 400 hours or more, given the amount of additional information to collect, necessary data collection and reporting software training, software enhancements, data quality checks, etc. We conservatively estimate the cost burden to be in the area of $10,000. This does not include the annual support cost of approximately $10,000 for the software utilized to generate the data in the CSBG report. (Based on our agency’s percentage of clients served by the statewide network supported by the software.)

3. The quality, utility, and clarity of the information to be collected; and

Specific comments speaking to these points will be made below on individual modules.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

Automating the collection process may be beneficial to the states in aggregating the data, but does not impact the data collection burden of any local agency. A significant improvement in data collection efforts at the local level would be facilitated by funding sources allowing for more flexibility in data collection and reporting systems, and urging vendors providing data collection systems to accommodate sharing information between systems so that local agencies can more easily collect unduplicated counts of individuals served. Much time and effort in collecting data could be saved by ensuring that current software programs, which are frequently proprietary and some cases mandated, are able to “talk” to data collection systems at the agency level, so that data does not have to be gathered from so many different places.

Comment 114 (Southwest Oklahoma Community Action ; Other/Unknown in OK)

Is this a valuable system? Southwest Oklahoma Community Action Group, Inc. recognizes the importance of accountability and continuous performance management improvement; however, the idea that a report that potentially triples the reporting requirements will somehow improve outcomes for customers is simply mistaken .First, all federal dollars require performance management and outcomes reporting. When Community Action Agencies also report this information in the CSBG IS report, it is in essence duplicating numbers. For instance, in our agency we were funded last year to serve 250 children with Head Start. Those children are being reported in the Program Information Report (PIR) to HHS, and they are also being reported in the CSBG IS report; hence duplicated reporting. Staff hours are being used to aggregate data in the format required by Head Start, as well as reporting in a manner proscribed by CSBG. Southwest Oklahoma Community Action received approximately $114,000 in CSBG funding last year. Each time the data reporting is increased, there is less funds for working to reduce the causes and effects of poverty and to help people achieve self-sufficiency. Secondly, the models that are followed do not support a whole client approach across a multi-year platform which is desperately needed in order to help people break the cycle of poverty. Having a job for six months does not indicate stability .Many people can work six months at a time, but they do not make strides because they are always changing jobs. Agencies should be finding solutions for why generations of families continue to need our services. Additionally, agencies have had to add in the requirements of Organizational Standards for data tracking_ This alone is cumbersome due to the number of standards and the varying time frames. One can quickly become out of compliance just by scheduling difficulties or staff turnover. Some local agencies have gone so far as to discuss relinquishing their CSBG dollars. Our opinion is CSBG is the heart of the agency; it is what makes our organization a Community Action. Yet, we have growing concerns about the utilization of CSBG for more and more recordkeeping and less work in the communities we serve.

2. Is the cost estimate that HHS provided correct? The estimated time burden of 242 hours per sub-grantee (local CAA) is significantly under estimated. Based on our estimation, the burden to our local agency would be approximately an additional 1,000 hours for the ROMA Next Generation reporting and the Organizational Standards maintenance. This estimate does not include time that other programs spend collecting and providing information for the CSBG reporting. It is anticipated that the cost of the 1,000 hours will reach as much as $35,000 (salaries and fringe) depending on which staff members are tasked with the additional reporting. This is an additional 30% of our organization's CSBG funds.

It does not improve an agency's performance to report outcomes in multiple reporting formats (CSBG, Head Start, Weatherization, etc.) .This are already being done in grants management. What makes agencies more performance driven is how they are collaborating in the community to solve the problems of that community, what is being done that is unique and successful, the ability to think creatively and outside of the box. CSBG has always been the seed money to either start new things or to address things that didn't have funding or to help augment programs to improve success. As always, Southwest Oklahoma Community Action Group, Inc. will strive to meet all requirements; however, it must be noted that each time there are more added requirements, service to the community is reduced. We appreciate the opportunity to comment and provide feedback.

Comment 115 (State Of North Carolina; State in NC)

No Content Unrelated to a Module for this comment.

Comment 117 (Texas Department Of Housing And Community Affairs ; State in TX)

In the Department’s original feedback, we stressed the importance of developing straight forward and simple forms. The current draft documents reflects a high level planning approach to reporting that needs to be simplified to a level where all CAA’s can effectively participate and respond. Assessment of the design and structure of the proposed reporting of performance for community action agencies must include a critical evaluation of the volume of data requested. The proposed reporting is so significantly complex and different that most CAAs will face big challenges in effective implementation this new and extremely complicated system of reporting. The burden of reporting more information, difficulty in interpreting and understanding what needs to be collected and an increase in time spent on reporting, may negatively impact the level of CAA participation in all the areas of reporting.

In finalizing the CSBG Annual Report, thoughtful consideration needs to be given to the capacity of the network. The CSBG network staff have diverse levels of education and skills that are inherent characteristics of local community action agencies. Many eligible entities operate with limited budgets, which impacts the staff that they are able to hire and retain. A very small percentage of eligible entities have a staff person with a planning background or who can focus only on reporting. The CSBG Coordinator/Director usually is the individual who oversees the collection and reporting of data, but caseworkers have to understand what information has to be collected (NPIs, etc.). The CSBG network will be challenged and may be overwhelmed by the new reporting if it is not shortened and simplified.

We must also realistically think about what the completed CSBG Annual Report will demonstrate about the work our network does. If the new report collects thousands of data points and the vast majority of CAAs only report on a small percentage of the activities described in Module 3 or Module 4, this will not help to illustrate the good work that the CSBG network does.

Although the CSBG National Information Systems (IS) Survey is being merged with the CSBG Annual Report which will report on State Accountability measures, the sections that represent the IS Survey have become too complex. So much new information is being collected and the volume of information much greater, that the bigger message of what the Network accomplishes will be lost. Reporting performance that is determined from data derived from inter-related activity or activity from other funding sources is very complex and time consuming. Although CSBG eligible entities only need to report on Community Level Initiatives and Individual and Family NPIs that they have activities on, the forms are complex and difficult to follow and reported sections require the reporting of additional related information.

Comment 118 (The State Of New York Community Action Association ; State Association in NY)

Question 1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

The proposed collection of information includes some useful elements but in whole is quite burdensome and includes information that is of limited use and well beyond the parameters of the Community Services Block Grant (CSBG.) Suggested modifications which could result in a useful and effective system of information collection are noted below.

1. Module 1: Allow CSBG State Lead Agencies/grantees to report on the eligible entities’ compliance with standards at a range of levels beyond 100%, for example, 50%, 75% 90% and 100%. The Organizational Standards were intended to promote excellence not merely the lowest level of compliance. It is therefore reasonable and important that the metrics include those agencies that meet less than 100% to provide a more accurate assessment of the training and technical assistance needs of eligible entities and to track CAA’s progress in meeting the Standards over time. Requiring that only 100% compliance be reported undermines the intention of the standards to promote excellence and is of limited to utility for the CAAs, State lead agencies and NASCSP and OCS.

2. Module 3: Make the reporting of community level data on social indicators optional and provide flexibility in defining which indicators to use and report. The inclusion of Community Outcome data with percentages or rates provides little meaningful data on the initiatives undertaken by local agencies. Their inclusion is inappropriate and will provide meaningless data when aggregated beyond individual initiatives.

3. Module 3: Collective Impact is only one of a variety of options for community-level work and should therefore be removed. As one in a menu of options, collective Impact should be removed as including it suggests a strong preference for this approach as opposed to other possible community-level strategies.

4. Module 4: Remove the Characteristics for NEW Individuals and Households Report. There is no definition of “New” included here nor is there a widely accepted standard for identifying a “new” customer. For example when is a customer who receives multiple services over many years “New”? And what utility does collecting this data have for state agencies or OCS when it is rolled up and aggregated? The purpose identified by OCS for the collection of this data is to inform the needs assessment and program development process. Agencies already have a process for identifying needs and developing services with little to be gained by expending the time and resources needed to collect this data. Furthermore, many agencies lack the technical capacity and resources to collect and analyze this data and many are challenged by the barriers created by the mandated use of multiple, often incompatible, reporting systems.

5. Module 4: The Stability Indicators included here are neither practical nor useful and should be removed. One option included here uses a tool for defining financial stability that is based on assumptions drawn from the general population and is therefore of very limited utility when applied to the low income people served by CAAs. Another option duplicates data already collected.

6. Reinstate the ROMA Goal: Agencies Increase their capacity to achieve results. This goal reflects the vital role of the national network of local CAAs and there is little sound reason to remove it. Agency capacity is critical to the success of the services provided and should be identified as such in the National Goals. Inclusion of Agency Capacity is also important to highlight the ongoing need for training, technical assistance and other capacity building services for the CAA network.

Question 2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.

While this is a question that can only be answered by individual CAAs, this Association is concerned about the burden of this enormous increase in the collection of information with no additional resources provided to meet this requirement. The increase in the data elements included in the proposed CSBG Annual Report from 300 to more than 1,000 poses potentially great burdens with no additional resources. This is particularly problematic given the challenges already faced by agencies that are required to collect and report data using multiple, often incompatible data systems.

It should also be noted that the on-line system described by OCS in the Dear Colleague Letter of June 17, 2016, may be of benefit to State Lead Agencies but does little to address the issues that eligible entities face in tracking and reporting data for multiple agencies using multiple information and reporting systems.

Question 3: The quality, utility, and clarity of the information to be collected.

A number of the elements to be collected are primarily useful as components of community-wide scorecards and have very limited utility or are not available for single agency reporting or as data to describe the work done in the smaller geographic areas served by local agencies, particularly regarding community-level work. The use of such data to assess and inform individual agency outcomes is at best limited and may in fact be misleading if used to compare agency outcomes with community indicators.

· Module 3, Section A: The indicators used here are to report community-level data to States and by States to OCS and will be of limited utility. The available community-level data often do not match the geographic area and population targeted by the agency, for example by census track or age. It is also extremely difficult to assess the impact of an agency’s intervention as part of a community-level initiative since many external factors influence the impact of such an undertaking and could erase or minimize the impact of community-level initiatives. Evaluating the outcomes of community-level initiatives requires a formal program evaluation and cannot be accomplished solely through the use of data collected in the Annual Report.

· Module 4, Section A: Characteristics for NEW Individuals and Households - Data Entry Form should be removed. As noted above the collection of this information is both difficult and costly and of little use at the local level. This data does not include information about the needs and challenges facing “new” customers and does little to help agencies assess whether they are meeting their needs. Additionally, this data fails to account for the many “one time” customers who access services such as LIHEAP and thus makes interpretation of the data difficult and inaccurate. Further, as previously noted, because many agencies are required to use multiple, incompatible data collection and reporting systems, this will pose a significant burden on agencies with no resources available to support this work.

Question 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

As previously noted, the on-line system referenced by OCS in the June 17, 2016 Dear Colleague Letter will do little to help agencies track customers, services and outcomes. The online automated system may be an asset to State Lead Agencies, but offers little for local Community Action Agencies which will still be required to track and report data using multiple information management systems with different, often incompatible requirements. It is unlikely that the State Lead Agencies will have sufficient resources to enhance the network’s data collection capacity nor will the proposed on-line system address the need for improved information management capacity as required by ROMA Next Gen. To minimize the burden and maximize the utility of the data collected, the federal government should assess its data collection requirements and find a way to use information management systems that can be integrated to meet the needs of the multiple government funding and regulatory agencies.

Comment 119 (TN Department Human Services ; State in TN)

Our biggest concern is the timing of the rollout of the CSBG Annual Report. It is believed that if all moves forward as expected that the CSBG Annual Report could be finalized as early as October 2016, but may not be approved and finalized until mid-winter. This does not allow sufficient time for our state to train each of our eligible entities, and for agencies to submit Community Action Plans based upon these new reporting standards, especially since Tennessee is one of five (5) states obligated to begin the reporting process beginning this coming July 2017 (since we operate on a State Fiscal Year).

October 2016-January 2017 CSBG Annual Report Finalized

January-March 2017 Develop and provide training to eligible entities

March-April 2017 Eligible entities receive Community Action Plans to write

May 2017 Eligible entities submit Community Action Plans for approval

July 2017 Eligible entities begin tracking outcomes based upon new CSBG Annual Reporting

As a network we depend upon our national partners to lead us in providing training and technical assistance. I am not sure they will be prepared to provide the training and technical assistance needed in time, nor will we have the ability to develop quality training given the short turnaround. In addition, mother nature is our enemy this time of year (January-March 2017). Besides the fact that we do not have any funding allocated to providing training and technical assistance. Our state has a 95% pass through of funds, and it would take some time to negotiate with eligible entities an alternative funding schedule to support funding for the required training and technical assistance.

On top of that we do not have any kind of data base ourselves. Seventeen (17) of the twenty (20) agencies use THO Software, one (1) agency uses CAPTAIN, one (1) agency uses the e-logic system and one (1) agency uses excel like us. At this time, without knowing the database requirements established through ACF and OLDC the burden is enormous to get all twenty (20) agencies to come to an agreement on what that system should be, or have the expertise of what that system should look like, without knowing that the CSBG Annual Report will be finalized and what those requirements will look like. Again, the short turnaround is weighing heavily on us. We don’t want to develop a system that ultimately has to be scrapped because we cannot get the information we need from eligible entities and we do not have the ability to upload information into OLDC as requested. Again, we do not have the funding allocated to establishing a data base. Our state has a 95% pass through of funds, and it would take time to negotiate with eligible entities an alternative funding schedule to support funding for a state(wide) system.

Finally, we have concerns with the overall burden of time it will take for agencies to track and report the required information. It already takes us nearly three months, from January 1-March 31 every year for agencies to submit their reports (due December 30), our department to input the data into the database, and work with the agencies to address any discrepancies. Knowing that the reporting standards have been expanded I am sure that those estimates would increase, and not increase…unless we had a data base that was could pull information directly from the agencies and be reported directly through OLDC.

Comment 121 (Total Action For Progress ; Other/Unknown in VA)

As mentioned above, TAP is fully committed to reporting accurate, relevant, and significant data that informs OCS, Virginia's Office of Volunteer and Community

Services (OVCS), and other funders, clients, board members, and other stakeholders of the progress TAP is making in helping individuals, families, and communities become strong and stable and to flourish. However, TAP has several concerns about the proposed CSBG Annual Report requirements:

In order to do this type of data collection and tracking, it will require newer and different resources and skills than our agency — and other CAPs — have.

We will need to develop and maintain a data analysis department whose sole focus is tracking, maintaining, analyzing, reporting and ensuring the quality of the data entered.

CSBG is such a small piece of our budget — our programs already must report to their primary funders, many of whom are federal agencies. We are being asked to coordinate our data reporting when even within departments of these agencies, there are multiple data systems which aren't coordinated. O We asks that we be able to use other reports that federal agencies already
require of us.

O If this is not possible, we ask that OCS wait until data systems up to the task have been designed, made available at a cost that isn't prohibitive, and been tested in a pilot program before requiring that CAPs use a new system.

The burden of completing the current funding application and reporting (particularly the fourth quarter report) is already crippling.

In order for the Board to be able to ingest, understand, and provide oversight, they need a report that ALL members of the Board will be able to use. A snapshot of important outcomes is necessary for this purpose.

Overall, TAP has the following recommendations: We recommend that OCS reduce the number of required components to the most relevant outcomes and outputs and select performance indicators that are relevant and can be measured and reported at the state, regional, and local levels. Further, it is not reasonable to think that CAP agencies will be able to conform to all the requirements by October 2017. We will need years to build our capacity to do so, including planning, budgeting, and training.

2. Is the cost estimate HHS provided correct?

HHS has estimated that it would take each sub-grantee (CAP agency) 242 hours at an as yet unspecified cost to the agency. After careful consideration, TAP estimates conservatively that it would take at least 5155 hours to fully comply with the
requirements, which is approximately 2473 more staff hours than it takes to comply with current requirements. Further, it will cost at least $296,925 to comply with the reporting requirements, which is more than half of the CSBG funding that TAP receives annually, funding intended to pay for services and administrative supports.

While some of the estimated cost is related to purchase and/or development of a data management system for the agency, most of the costs would be ongoing annual expenses and are broken down below:

ACTIVITY

Amount of time it takes for staff to collect information not required by other funders

Amount of time workers will spend entering data or transferring it from another program's software Amount of time required for each worker to be trained and refreshed on the skills of data entry or acquisition Amount of time our team will spend on data quality assurance, as well as quality assurance on the final report

Amount of time staff will spend tracking individuals and families, tracking progress of community projects, and recording the progress in the format required Amount of time spent tracking volunteers, board members and their commitments to organizations and projects that are not run by your Community Action Agency Amount of time spent on new reports to the state required for each of the 50 organizational standards, with additional analysis required on any standards that your agency has not achieved in full

Placement of data into the state reporting software, including checking and answers to the narratives required by the state for module 1, as well as the local information on funds, people and projects Contracted professional services IT and

COST JUSTIFICATION and

ESTIMATED HOURS

25 programs x 10 hours per program x $20/hour (average salary and benefits)

5 minutes per client served x

5700 clients

25 hours per person x 100 people x 20/hour

Data: 64 hours x $50/hour Final report: 20 hours x

$50/hour

35 hours x 8 staff x 4 quarters x

$50/hour

Salary and benefits for a full-time staff member

140 hours x $50

40 hours x $50/hour

120 contracted hours x

TOTAL

COST

5000

28,500 50,000

4200

56,000 50,000

7000

2000

13,200

Maintaining the software to keep and integrate mandated data, and to track our projects and our progress on or anizational standards Purchasing new software Supporting and training staff who use the software ROMA training — staff time

ROMA training — training cost

External data acquisition (surveys, demographics, etc.)

$110/hour
Actual cost for current software

One-time cost

Trainer contract

80 hours per staff person x 3 staff x 20/hour

$775 per person to go to the training 26 weeks x 1 staff x $20/hr. x 40 hours

3100

40,000 10,000

4800

2325

20,800

Comment 122 (Trehab Community Action Agency ; Other/Unknown in PA)

Trehab is a Community Action Agency that covers 6 counties in Northeast Pennsylvania. Our Agency is committed to act as a service provider and advocate for the poor, unemployed, underemployed, elderly and other groups at risk.

Within the parameter of The Agency budget, the reporting requirements cause a large; personnel financial burden A more simplified system could easily relay client reporting requirements. The time and financial burden take away from efforts to provide essential services.

The time burden and costs to implement system estimated annual cost is approx. $108,810 .00.

Comment 123 (Tri County Community Action ; Other/Unknown in MN)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The measure of 100% being used to define the performance of the local agency is unrealistic and impractical due to agencies implementing different programs based on the needs of their local communities. If the collection of information is to be used for agency performance, a range of meeting the standards should be used such as 100%, 90%, and 80% and so on to allow agencies a scale to improve rather than 100% pass or fail.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

It will be difficult at best to determine the agency’s burden of the proposed collection of information other than it will increase dramatically even with a proposed on-line data collection system. While it may appear that an on-line data collection system will reduce collection and improve tracking, the time to develop, test and train on a new system will be time consuming at best due to local agencies needing to track and report required data for multiple funding sources.

3. The quality, utility, and clarity of the information to be collected; and

Data collected by rural or smaller geographic area agencies will not be the same as data collection in larger, urban areas. Smaller agency data and accomplishments will be skewed and unclear in the overall data. Outcomes seen as positive improvements in a smaller community will not be viewed as such due to the smaller amount of population/numbers of people affected by the change.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

An on-line system will most likely not minimize the workload of the local agency as much as it would a larger state agency. Local agencies run several different programs/grants which often require different data tracking requirements as determined by the funding sources. Local agencies will not have funding or capacity in order to improve management information systems and most state agencies do not have funding to combine collection systems.

Comment 124 (U.S Department Of Health And Human Service ; Eligible Entity/Local Agency in DC)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility:

The proper performance of the functions of the eligible entities are currently assessed annually by our State CSBG Office through the CSBG required onsite monitoring reviews, CSBG Organizational Standards assessments, State Accountability Measures, CSBG State Plan and Application requirements, and monitoring for adherence to the ROMA performance management cycle of assessment, planning, implementation, achievement of results and evaluation.

Although some modules and sections of the proposed CSBG Annual Report would provide useful or meaningful community, family, and individual outcomes and information, other areas, as described later in this document, would require information that is not necessary for the proper performance of the functions of the eligible entities.

The accuracy of the agency's estimate of the burden of the proposed collection of information:

In our opinion, the burden hours estimated for the State CSBG Office (164 hours) is significantly underestimated. Our estimation, based on the processes we conduct in order to complete the current annual report (CSBG-IS), easily exceeds 270 hours. Those processes are:

Preparing annual report forms, instructions, and guidance o (24 hours)

Providing an annual training to the 17 eligible entities in our state o (4 hours)

Ongoing training and technical assistance to the 17 eligible entities o (30 hours) approximately 10 hours a month for 3 months

Individualized training and technical assistance to new eligible entity staff o (12 hours) 3 staff @ 4 hours

Collecting the data and information from our 17 eligible entities o (16 hours)

Reviewing and conducting data quality checks on all the data and information submitted by the 17 eligible entities in order to assure accurate data o (136 hours) 8 hours per agency

Collecting and preparing the state data and information for the annual report o (8 hours)

Transferring the eligible entities and state data into the reporting system o (24 hours)

Inputting our data quality notes into the reporting system o (16 hours)

Given the increase in the amount of data and information the proposed CSBG Annual Report would require, this can only add to our burden; and these new requirements come with no new administrative resources. The following are sections in the proposed report that contain new reporting requirements. It is difficult to estimate the required hours for these new requirements, but they would add burden hours to our estimate (These new requirements are not included in our estimated burden hours listed above):

Module 1 -Sections 82, B3, B7 o Analysis on trends and results from available ROMA data could be a substantial increase in burden (see B7)
Module I — Sections G, H, and I

Module 3—

Section A (Community Level Initiatives in Progress)

Module 3 — Section A (Community Level Strategies)

Module 4 — Section A (NEW Characteristics Report

Module 4 — Section D (Individual and Family Services)
A number of State CSBG Offices, including ours, are allotted less than 5% of their state's annual CSBG allotment for administrative costs, and do not receive any of the state's CSBG funds for remainder/discretionary projects. The allotment and availability of CSBG funds to State CSBG Offices must be considered when reviewing and deciding on how the new requirements and proposed changes presented in the proposed report would affect the burden of State CSBG Offices.

The quality, utility, and clarity of the information to be collected:

QUALITY

Our State CSBG Office and our eligible entities agree that the states and eligible entities should report their performance measures annually to OCS. We also agree that some of the new requirements and proposed changes would be improvements to the current CSBG ROMA performance measurement system. These new requirements and proposed changes include:

State CSBG Offices reporting their statewide goals and accomplishments

State CSBG Offices reporting on their CSBG Organizational Standards assessments State CSBG Offices reporting on their monitoring of eligible entities

The proposed changes to reporting Eligible Entity Expenditures, Capacity, and Resources

The new Community Level Strategies requirements

The proposed changes to the ALL Characteristics Report

The new Individual and Family Services requirements

The proposed changes to the Individual and Family NPls

However, we also agree that some of the new requirements and proposed changes would not improve the system, and would likely add to the burden of State CSBG Offices and eligible entities. Some of those concerns are:

Summary Analysis (Module 1, Section B7)

State Linkages and Communications (Module I, Section G)

State and Eligible Entity Analysis of Data (Module I, Section 13)



State Feedback on Data Collection, Analysis, and Reporting (Module I, Section 14)

NEW Characteristics Report (Module 4, Section A)

The new requirement that would require eligible entities to report Community Level Initiative baseline numbers (Module 3, Sections A and B)
The proposed changes that would require eligible entities to report their targets for all their NPls (Module 3 and 4)

Our comments and feedback regarding these concerns can be found on pages 5 and 6, under ADDITIONAL FEEDBACK AROUND DATA ELEMENTS

CLARITY

In general, we feel that the data and information to be collected in the proposed CSBG Annual Report is mostly clear. Our concerns around clarity are:

Module 3 (Section A) — This is a new section and it has a number of new data elements and requests information that is unfamiliar to us. Our State CSBG Office and eligible entities would need training and technical assistance specific to this module prior to implementation.

Implementation Timeline — We understand that full reporting would begin in FY 2018. However, we understand that State CSBG Offices would be reporting on some of the new requirements and proposed changes as soon as March 31, 2017. We do not believe this would give us a sufficient amount of time to prepare and collect a lot of that information.

If the proposed CSBG Annual Report is approved and OCS provides additional guidance, an instruction manual with definitions, and training and technical assistance prior to full implementation, there shouldn't be any major issues around clarity and consistency.

UTILITY

The sections for collecting and reporting State CSBG Office and eligible entity performance measures would be useful. However, we don't believe the proposed CSBG Annual Report would be a quality tool for eligible entities to use for planning and analysis. Our eligible entities stated that they use their client and community needs assessment data, program specific data, and other local data and information for program planning and implementation; rarely do they use the data and information from their Community Action NPI Report when they conduct planning at their agency.
Also, as stated in question #1 under OCS REQUESTED FEEDBACK, we currently assess our eligible entities annually through the CSBG required onsite monitoring reviews, CSBG
Organizational Standards assessments, State Accountability Measures, CSBG State Plan and Application requirements, and adherence to the ROMA performance management cycle of assessment, planning, implementation, achievement of results and evaluation. Therefore, to be useful, the proposed report should only be a report for State CSBG Offices and the eligible entities to report their performance measures.

Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology:

The proposed CSBG Annual Report should be a report for State CSBG Offices and the eligible entities to report their performance measures. The report does not need to be, and will not be used by our eligible entities, as a planning and analysis tool for continuous quality improvement. Eligible entities use their client and community needs assessment data, program specific data, and other local data and information for program planning and implementation; rarely do they use the data and information from their Community Action NPIs Report for planning and analysis. Eligible entities already conduct ROMA planning and analysis for Eligible entities already conduct ROMA planning and analysis for continuous quality improvement and are monitored by our State CSBG Office for those purposes. Therefore, to minimize burden, we suggest the proposed report focus solely on collecting meaningful community, family, and individual outcomes and information from eligible entities, and the necessary performance measures and information from State CSBG Offices.

Comment 125 (Vermont Community Action Partnership ; Other/Unknown in VT)

First, we want to recognize the incredible amount of thought, effort, collaboration, communication, and creativity that went into the development of this resulting process and document proposed to reflect the next generation of the ROMA framework. We also want to acknowledge that a great deal of the proposed process and document is positive, constructive, and has the potential to be of significant benefit to the Community Action network and to our efforts to continue planning, implementing, monitoring, and evaluating the ongoing War on Poverty and to inform the public about its value and effectiveness in supporting and lifting up millions of Americans struggling to survive and create a better life.

That said, we also feel the need to convey some concerns regarding a number of aspects of the proposed process and document which we feel will diminish its effectiveness and actually risk interfering with the work of our agencies by making the administrative requirements imposed on us significantly, and unnecessarily, more burdensome, without commensurate benefits resulting from that increased burden.

Our concerns are largely focused in the following 5 key areas:

* Module 1: Reporting on CSBG Organizational Standards and Technical Assistance Plans / Quality Improvement Plans for local agencies with unmet standards.

* Module 3: Community Outcome Indicators and Collective Impact

* Module 4: Characteristics for New Individuals and Households

* Module 4: Stability Measures / Indicators

* Modules 3 & 4: ROMA Goals – Lack of inclusion of an Agency Capacity-Building Goal Q. 1: Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

While there are elements of data collection that that may be of value, there are a number of areas, as indicated below, where the collection of information does not meet the threshold of necessity or utility. We have to look at the proposed package as a whole, and when we view it that way, it is far too burdensome, contains reporting tools that will provide information of little or no utility, and is outside the reach of the Community Services Block Grant (CSBG). However, with some modifications, a system for collection of information could be developed that has practical utility and can be used to improve the performance management of our agencies. We are in substantial agreement with the Community Action Partnership regarding many of their recommendations for improving the proposed package, as indicated below, to achieve an effective level of workability and utility.

1. Module 1: Modify the options State CSBG Lead Agencies and grantees have when reporting to OCS on CSBG Organizational Standards, to allow for a broader range of responses. As noted below, there are significant challenges with the way this information is being reported that misinterprets and misuses the Organizational Standards. Expecting constant perfection is unrealistic and does not have practical utility for either local agencies or the State CSBG Lead Agency (LAs). As proposed, the single measure the State Agencies will report on is the number of Eligible Entities (EEs) that meet 100% of the CSBG Organizational Standards. To make this measure practical and realistic, and to increase the usefulness of the information, LAs should have the option to report the number of agencies at a range of thresholds that still indicate strong performance as well as allow for a more accurate reflection of the EEs’ training and technical assistance needs. For example, reporting the number of local agencies that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of the standards will allow for the collection of more accurate information, better tracking of results over time, and better, more efficient investment of training and technical assistance resources.

2. Module 4: Remove the Characteristics for New Individuals and Households Report. This report has very little if any practical utility at the local level in most cases, and will be a significant net drain on agency resources, given the time and expense required to create it at each local agency. This is not necessary for agency performance (and may in fact contribute to diminishing it), and the intentions outlined by OCS for its use are not reflective of how local agencies and states look at their communities to assess needs, develop programs, and report progress. It also lacks utility at the state level given that data such as this rolled up to the state level will often be skewed by large population centers and will therefore decrease the practical utility of the data even further. In addition, many agencies, including those in Vermont, either lack management information systems with the capacity to disaggregate this type of data from its overall customer database, or struggle to integrate the required data given their use of multiple (and often incompatible) reporting systems required by our disparate funding sources. Moreover, since our agencies serve many families over time, with no definition of “new” being provided for review, the CSBG network will be unable to implement this report consistently. Defining an individual or family as new becomes challenging when they may receive different services over many years (e.g. Head Start / Early HS, child care, afterschool programs, youth services, job training / readiness, food, fuel, homelessness prevention, small business support, financial literacy, IDAs, income tax assistance, senior services, etc.).

3. Module 3: Remove the Community Outcome Indicators that include percent’s or rates. Their inclusion is inappropriate and will provide meaningless or even misleading data when aggregated beyond individual initiatives. Such data has validity for geographic areas and populations much larger than those addressed or targeted by local agency services and initiatives. This validity is lost when the scale is smaller and generally accepted data comparison points become unavailable. Having individual agencies report on rate change at the community level on social indicators does not provide information that is usable. In fact, it is more likely to end up being detrimental to our purposes, since it will more often under-report the actual outcomes and impact on the people the agencies are actually serving when they are compared to an inappropriately large community-wide population that most agencies have no realistic capacity to make a measurable impact on. It is much more appropriate and useful to give agencies the option of reporting outcome data and the flexibility to define which indicators they use.

4. Module 3: Remove Collective Impact from all reports. Collective Impact is simply one set of strategies for doing community-level work, and while EEs are engaged in such efforts, privileging one approach over others, given the broader range of options for doing community-level work, is inappropriate and unnecessary. We are suggesting that Collective Impact is not a worthwhile strategy for many agencies to engage in. We’re just recognizing the reality that relatively few of us have sufficient resources and/or are in a position to mount an effective initiative of that kind; consequently, making it an expectation in the core reporting tool will likely create the unfortunate perception that the majority of agencies are failing to meet a key expectation and thus diminish the general perception of the entire network’s effectiveness. This should be removed and instead encouraged as part of training and technical assistance initiatives.

5. Module 4: Remove the Stability Indicators. The indicators selected and included in the clearance package are not helpful or practical when reporting customers stabilized as a result of the EEs’ work. One option duplicates data already collected and the second uses a tool that defines financial stability based on a construct the uses assumptions drawn from the general population, not those of low-income individuals and families. Because financial well-being means something very different for the low-income population in comparison to the general population, use of the scale will produce data of limited utility and, as with Collective Impact, may actually distort the perception of our effectiveness.

6. Include a 4th ROMA Goal: Agencies Increase their Capacity to Achieve Results. Re-instate this current national goal that addresses the critical role the network of local agencies serving the vast majority of the country plays in addressing poverty. The stated reasons for proposing to eliminate this goal have been inadequate, missing the need to focus efforts and resources on continuous improvement. Because agency capacity is critical to the ultimate success of our programs, it is important to clearly state this objective in the overall National Goals. This also highlights the broader need of the network for ongoing training, technical assistance, and general capacity-building supports.

Q. 2: The accuracy of the agency’s estimate of the burden of the proposed collection of information.

It has proven to be extremely difficult, if not impossible, to come up with a realistic estimate of our agencies’ time and cost burden to implement the proposed new ROMA-NG package, since there is so much of it that is untried and untested, and largely because the new technology needs and costs – in agency personnel, IT consultants, software and equipment – that will be needed to meet the expanded data collection, analysis, and reporting requirements of ROMA-NG are very significant and beyond our current capacity to make even a reasonably accurate rough estimate. What we know for certain is that, since no additional resources will be provided to EEs to build our capacity to meet these new and expanded requirements, the burden will undoubtedly be far greater than most if not all of we can absorb within our existing infrastructure and resources. Our concerns are in part based on the following details:

1. The current IS survey includes around 300 data elements to report. The proposed CSBG Annual Report includes more than 1,000. This increased burden of reporting comes with no new resources.

2. Most if not all of our local agencies lack information management systems with the capacity to provide the required data; we have been struggling to find or develop adequate, reliable, and user-friendly systems just to deal with the current requirements for generating comprehensive, agency- wide, unduplicated client data. Based on our collective experience to-date, the data fields required for ROMA–NG reporting are likely not to match those currently used by our agencies, and will almost certainly mean additional significant and complex negotiations, time and costs associated with integrating our required proprietary data management systems (e.g., HMIS, Hancock for Weatherization, PROMIS and other Head Start software, etc.) in order to produce agency-wide unduplicated data.

3. Despite OCS’s claim in its June 17th Dear Colleague Letter that the burden for local agencies will be decreased by the implementation of the automated Online Data Collection system (OLDC), it’s not at all clear how that will occur. It may in fact benefit State Lead Agencies, but will most likely not reduce the time or cost burden for local agencies. We will still need systems that track customers, services, and outcomes, and given the specific requirements of many funding sources (including other federal HHS divisions), that will mean additional and often separate data collection systems. To comply with the draft CSBG Annual Report local agencies will need to modify, expand, and in some cases replace their existing systems. State LAs do not have the funding or capacity, for the most part, to create systems or support local agencies to develop systems that will streamline or decrease the burden imposed by the proposed package; and what funding they do have is from the same pool of resources (e.g., CSBG Discretionary funds) that is now used to support expanded and new, innovative programs and/or emergency needs of the local agencies, thus reducing or eliminating current opportunities.

Based on all of the above concerns, although we are unable to come up with specific estimates of the additional time and cost that for our local agency will need to meet the proposed ROMA-NG data requirements, we feel certain that OCS’s estimated time burden of 242 hours per local Community Action Agency is significantly underestimated. We project that at a minimum, the burden to our local agencies in Vermont would be double or triple that amount, if not even more. The cost of this time burden is impossible for us to estimate at this time, but we would like to note for the record that the additional hours of staff time is only one element of the additional cost, which will include significant expenditures for IT consultants, software purchase and/or modifications, and equipment to enable us to meet the new requirements.

Q. 3: The quality, utility, and clarity of the information to be collected.

As noted below in the detailed comments by module, several elements of data to be collected under this proposal have no meaning at the state and federal levels, and only have value, meaning, and usefulness at the local level.

1. Several of the elements to be collected are not available in smaller geographic areas and among sub-populations usually targeted by local agencies in our community work and, if created by local agencies, lack statistical validity. The social and population level indicators being requested are not meant for single agency reporting and are best used as part of a community needs assessment. For example, comparing overall percentages of community indicators (e.g. the unemployment rate, violent crime rate) to individual local agencies’ outcomes is misleading and is likely to have the unintended consequence of making it look like our services are ineffective because they’re not producing change on an impossibly-expected scale.

2. Module 4, Section A: Characteristics for New Individuals and Households – This Data Entry Form is an unnecessary document that will not produce quality data; it will have little usefulness at the local level; and it lends itself to meaningless and misleading data at state and national levels; consequently, it should be removed. Because demographic data alone does not include information about the needs of new customers, this information fails to help agencies determine if they are addressing issues identified in their community assessments. Furthermore, the large number of “one time” customers in programs such as LIHEAP will skew any meaningful interpretation of the data. Aside from the challenge of defining who constitutes a new customer, agencies that use multiple reporting systems will also have the added, and extremely challenging, burden of tracking new customers across all programs and services.

3. Module 3, Section A – Many of the indicators used here are inappropriate for reporting by local sub-grantees to States and by States to OCS and will produce data of limited utility, for a number of reasons. First, community-level data on indicators is not always available at the level (e.g. regional or zip code/census tract) or population (e.g. low-income children aged 0-5) that matches the area and population targeted by the agency. Second, community-level indicators are influenced by a wide number of sources that would most likely overwhelm the impact of even successful community initiatives, a fact that significantly limits their utility for providing meaningful information about outcomes that doesn’t under-report the real impact on the people served by our agencies. Third, any meaningful data about the outcomes of community-level initiatives would require a rigorous program evaluation and could not be determined on the data collected through the Annual Report alone.

Q. 4: Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

OCS and other federal agencies provide funding to EEs that require separate systems and prohibit the aggregation of data. To minimize the burden on respondents, DHHS should examine its own practices to ensure that all DHHS-funded programs in the field can use data collection systems that can be easily integrated and “talk” to one another to produce reports that meet the needs of local agencies, states, and federal agencies. OCS should also seriously consider a number of options for better supporting local agencies’ efforts to meet the data collection requirements, including: Providing funding for acquisition, development, and/or modification of integrated automated data systems; support the development of a national data management software application that will be provided to local agencies to use for this purpose; and/or coordinate the development of uniform specifications for data management systems that meet the ROMA-NG requirements and which all software vendors will be required to meet in order for local agencies to use them for this purpose.

Comment 126 (Vernon Community Action Council, Inc.; Eligible Entity/Local Agency in LA)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

It is our belief that the collection of information in relation to CSBG activities is vitally important as it is the foundation for which we continue to support the War on Poverty! If we do not have substantive data to support the need for our programs and services we will continue to experience status level funding and/or reduction in funding on a national level.

BUT OCS should have introduced this report to local level agencies state-by-state several years before it planned to implement the report. Meetings should have been held in every
state with mandatory participation by all CSBG funded entities. Many local level agencies still have not reviewed this report nor are they aware of the future implications and how it will impact their operations.

The collection of the data for this report will be labor intensive and with status-quo funding and the reduction of CSBG staff nationwide it will overburden staff, administration and Board Directors. Further, the impact it will have on the state level CSBG staff will be significant.

In order for local level agencies to accurate compile and report the data required by this report additional staff will need to be employed, a significant amount of training will need to be provided to all CSBG personnel and the Board of Directors and a national database system for data entry will need to be implemented.

The question lies in where the funding for all of the identified endeavors will come from. Again, this report cannot be implemented by all local level entities for possibly several years down the road.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

No local level agencies will be able to implement the necessary structure and practices needed to accurate report data for this report without significant training and without restructuring.

3. The quality, utility, and clarity of the information to be collected; and

Unless all local level agencies fully understand the report, the data that they will be collecting, the type of data to be collected and the proper procedures for the collection of data the report will not be effective. Again, to implement this report to assure positive and effective results the next several years should be spent in training local level entities. The training should have mandatory attendance by Executive Directors and CSBG staff at a minimum.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.
The implementation of a national database system that will be utilized by all CSBG funded entities.

We have elected to not provide an individual comment on NPI’s as our comments cover the entirety of the report. If the information that we have been provided is accurate and we believe that it is CSBG entities will be required to report on an estimated 1,131 items versus the current 374 that are in the system!

It is estimated that we will spend 242 hours collecting and reporting the data! In order to accurately report the data our organization will have to increase the level of knowledge of all agency personnel to include those whose positions are not funded by CSBG. To do so CSBG funds that are already overburdened will have to be pulled from much needed line items and moved to training and travel. This quite possibly will result in a reduction of personnel and/or a reduction in hours for current personnel. Further, additional funded will need to be generated for current CSBG personnel to support time spent with data collection and reporting.
Further, we believe the report is too complex and will not accurately tell our story to our staff, Board Directors, community, etc. The current report has to be downsized and manipulated in order for us to report needed information excluding data that is beyond the scope of what our audience needs or wants.

Collection of data to tell the CSBG story is essential and very much needed though the process should have started with local level agencies. We have been told that recommendations from our national partners were not taken into consideration! A training plan for all local level agencies to implement the final report needs to be established immediately.

Comment 127 (Virginia Community Action Partnership ; State Association in VA)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility.

We remain concerned that there are a number of areas where proposed collection of information does not meet the standard of either necessity for the proper performance of agency functions or the standard of practical utility for the network. Many of these concerns were raised in comments from Virginia community action agencies in their preliminary comments, but these suggested changes were not reflected in the document that we are currently reviewing.

Regarding public community action agencies, many of these agencies sub-contract their CSBG funding to other service providers in their communities. In some cases, these sub grantees may not have the capacity or funds for data collection that would meet the requirements of ROMA Next Generation. We believe that this possible challenge should be addressed in the proposed reporting package.

The ROMA goal that recognized the importance of agency capacity has not been included in the proposed CSBG Annual Report formulation. We think that the goal of increasing agency capacity to achieve results should be addressed in the Annual Report. Expanding agency capacity is critical to the success of agency programs and services. Agency capacity is closely related to the important needs of training, technical assistance and overall capacity building for the community action network.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information.

OMB review of proposed regulations requires a systematic identification and weighing of the costs and benefits of the regulation. This should include the cost of data collection at each local agency as well as the cost of establishing or upgrading a data collection and management system at the state CSBG office level. It would also include a comparison of present and future costs with the anticipated benefits of the regulation. Cost-effectiveness analysis also seeks to determine how a given goal may be achieved at the least cost. Two things seem clear to us—there has not been a comparison of the current cost burden of existing ROMA/State Plan requirements so that OMB would have a baseline to work with, and the costs of compliance with the proposed protocol will have different costs in different states and different local agencies. This will largely depend upon the current state of data collection and management systems at each agency and in each state. This is a very critical area for local agencies and state CSBG offices that are dependent upon CSBG core funding for their operations. I am not aware of any proposal by the Obama Administration that would provide increased funding to local agencies or state CSBG offices to help meet these substantial costs. The Virginia community action network does not currently have a statewide client data system, and the cost of establishing such a system for all local agencies will be considerable. A small group of Virginia local agencies are getting ready to release an RFP and we will have a better sense of the cost of this system (both establishing the system and ongoing cost of administration) when we receive responses to the RFP.

At this time, we believe that some local agencies lack the management information systems with the capacity to provide the required data. The data fields provided for ROMA Next Generation reporting may not be the same as those used by these agencies at this time. In addition, agencies with a broad variety of programs and services use multiple reporting systems (some of which are incompatible) and the proposed reporting would add significantly to the time necessary to provide the required data.

The on-line automated system that will be used by state CSBG offices may assist State Lead Agencies but most likely will not benefit local community action agencies in terms of time or ease of access. Local community action agencies will need systems that track customers, services, and outcomes. Separate funding sources (including other U.S. Department of Health and Human Services funding) require additional and usually separate data collection systems. State CSBG Offices do not have the funding or capacity available to assist local agencies in creating systems or to support local agencies to develop systems that will ease the burden of time and cost expected to meet the proposed protocol.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

The only way that the burden could be minimized on local community action agencies is for funding assistance to be provided to local agencies to assist in the development of client-based data management systems for each agency. New state-level systems will not address the needs of local agencies and state agencies do not have the funding or capacity to assist agencies in developing these data systems.

Federal Office of Community Services and other federal government agencies could assist in easing the burden on local community action agencies by ensuring that all federally-funded programs may use data collection systems that may be easily integrated so that local community action agencies, state CSBG Offices and federal OCS may more easily produce reports that meet the needs of all parties. Having separate systems and not allowing aggregation of data is a major burden, particularly for community action agencies which generally have many programs and services dependent upon different sources/streams of federal, state and local government funding.

Comment 130 (Wisconsin Community Action Program Association ; State Association in WI)

To that end, while we are very respectful of the tremendous amount of work that has gone into the development of the ROMA: Next Generation standards, we feel it important to re-state those conclusions we have developed and shared in previous comment periods:

* The ponderous level of detail and structure proposed creates substantial concern as to its impact on the work of our Community Action Agencies, as well as on the financial ability to properly address its requirements.

* The proposed system will require substantial amounts of data to be collected annually in reports that will be much more expansive than currently. The proposal to collect data on both the community and individual level, creating two sets of data, only compounds this increase. We strongly feel the size of the reporting is not commensurate with the level of funding currently existing to cover these activities.

* All of Wisconsin’s CAAs have multiple funding sources. Each of these funding sources already requires separate reports specific to their programs. ROMA: Next Generation will require another layer of reporting, asking for data which is not likely to correspond with these other reporting requirements. The potential result: more expense, through having to hire additional staff or purchase software systems capable of providing sufficient tracking, and more time and resources targeted towards reporting instead of the provision of services.

The draft mandates the use of limited outcome indicators in order to standardize and collapse the rich variety of Community Action Agency achievements and projects into a national summary. This is very troubling; as ROMA: Next Generation winds its way through the development process towards a final iteration, it is critical that the final product supports and enhances the core principal of local control that defines and drives what is unique and effective about Community Action. Vigilance must be maintained so that changes do not serve to mandate a pre-determined menu of services which will mold activities and programs on the local level towards a centralized agenda.

Unfortunately, the draft still contains a massive amount of added work that is not going to produce the outcome information Wisconsin’s CAAs need to adequately assess and continually improve performance. Instead, our agencies will only have other, non-report-related tracking to perform. And these indicators, as suggested by the draft, will not even serve the ancillary function of telling our story to policy makers and the public in an effective and comprehensive manner.

All of us believe strongly in accountability; the value of what the Community Action network provides to low-income households and the communities in which they reside can only be enhanced by transparency in activities and accountability in outcomes. But the measures used to define accountability, while they should be strong, need to also be developed and implemented in the context of the uniquely local and flexible nature of Community Action Agencies and in the realization that escaping poverty is a complex task, unique to each individual striving for economic self-sufficiency

Detailed comments submitted by the Community Action Partnership are consistent with these beliefs and were developed and refined through arduous effort and substantial input from the network of the nation’s Community Action Agencies, including those of us in Wisconsin. We, thus, support them with great confidence and urge that their conclusions be incorporated wherever possible into the final rules.

Finally, a word about the apparently increasing importance related to the agency estimates of the burden of the proposed collection of information. Many of Wisconsin’s individual CAAs will provide comments on this burden as it applies to their agency. In general, for our network, it will be substantial. Many agencies lack management information systems with the capacity to provide the required data. Several of the elements to be collected, as described, are not available in smaller geographic areas. Several elements of data to be collected under the draft proposal have no meaning at the state and federal levels. There is also the danger that identifying a ‘burden’ will become a self-fulfilling prophecy and that outcomes will be measured and enforced based on a reasonable ‘burden’ that has nothing to do with the very inexact level of attention and service to an individual family being helped towards economic self-sufficiency.

We appreciate that there is a great desire to conclude the tremendous amount of work that has gone into developing these outcomes/standards by forging ahead in the remaining months of 2016 to a final product. That satisfaction, however, will be shown to be very temporary if the network is saddled with long-term problems and frustrations that have moved Community Action away from its roots and the successes that have come – and continue to come – from Community Action’s unique approach to fighting poverty. We urge HHS to take its time to get this right, even should that move into next year or the year after. The stakes are too important and the long-lasting impacts of these critical changes too impactful to do anything

Comment 131 (Wisconsin Department Of Children And Families; Eligible Entity/Local Agency in WI)

1. Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility

The proposed Annual Report overall makes good improvements to the National Performance Indicators (NPIs). I particularly like that outputs are separated from outcomes, and while I know that our local agencies are nervous about the community-level NPIs, they will provide much more consistent and clear data about the community-level work being done in our state.

2. The accuracy of the agency’s estimate of the burden of the proposed collection of information

I believe that OCS’s estimated annual burden of 164 hours for grantees (State offices) is accurate. The estimated 242 hours for the eligible entities seems accurate, but will also vary greatly depending on the size of an agency and how organized it is in terms of tracking performance data.

3. The quality, utility, and clarity of the information to be collected; and

Overall, the information collected looks fine and useful. I made comments about specific data elements below.

4. Ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology.

The planned on-line automated system that auto-populates data and allows local agencies to enter their own data at the states’ discretion would be helpful and reduce the workload for state CSBG offices, but will only have a negligible impact on reducing the eligible entities’ burden.
Module 1 - State Administration: (pages 2-22)

Module 1, Section B: (page # 5-6)

p. 6 – B.7 Summary Analysis: The first six bullet points all seem like they will be easy for states to answer and will provide useful information.

The last two bullet points appear to place new expectations on the State office that are not included in the State Accountability Measures. They may be problematic on two levels: 1) they would increase the workload for our state’s CSBG administration, which is currently handled by one staff person; and 2) our state’s network of eligible entities values local control and is not looking to the State for directives on their programming decisions.

“Based on analysis, what services, strategies, or administrative practices were identified to be shared across the State?” The passive voice makes it confusing. Is the State CSBG office charged with identifying best services, strategies, and administrative practices to share across the state? Is this a new requirement from OCS?

“Based on analysis of performance data, what changes will the State encourage and support with the local eligible entities?” Is this question referring to 5.Sii? from the State Accountability Measures, which requires States to provide individual written feedback on the each eligible entity’s performance in meeting ROMA goals, or does it refer to a new requirement?

Module 1, Section E: (page # 10-14)

p. 14 - E.9b. Carryover for this Fiscal Year: Wisconsin's CSBG contracts operate on the calendar year and the agencies have 90 days after the end of December to close out their contracts. Therefore, the State did not have the final data to report at the time the Annual Report is due at the end of March. Our office will be able to send the correct Carryover amount during April, a few weeks after the Annual Report is submitted.

Module 1, Section H: (page # 20-21)

p. 21 – H.6 Single Audit Review: Does “Audit Number” mean “Report ID”?

Module 1, Section I: (page # 22)

p. 22 – “State and Eligible Analysis of Data: Describe how the State validated that the eligible entities used data to improve service delivery.”

Would this be addressed by monitoring the agencies for CSBG Organizational Standard 9.3, which states, “The organization has presented to the governing board for review and action, at least within the 12 months, an analysis of the agency’s outcomes and any operational or strategic program adjustments and improvements identified as necessary.”?

What other expectations would there be relating to how the State validates that the eligible entities used data to improve service delivery?

Module 2 – Agency Expenditures, Capacity, and Resources: (pages 23-27)

Module 2, Section A: (page # 24)

p. 24 – Table 2 – Details on Agency Capacity Building Activities Funded by CSBG: I’d recommend including “Volunteer Coordination” and “Grant Writing” to the list of activities.

Module 2, Section B: (page # 25)

1. Hours of Agency Capacity Building - This will be very difficult to provide an accurate count of all hours spent in capacity building. Providing hours spent in training as in the current IS Survey is relatively straightforward, but counting all of the hour’s staff and board members spend in “planning and assessment” activities would be much more complicated and time consuming. For some staff members, that could be a majority of their work hours and for board members, much of their time in regular board meetings and committee meetings could be considered capacity building activities. This would also be very much dependent on how agencies interpret the terms “planning and assessment”. In addition to the burden of tracking these hours, the practical utility of this information is questionable.
4. Partnerships Developed to Expand Agency Capacity – It is unclear what agencies would be counting here. The top row says Partnerships, but the column header says it’s an unduplicated count of organizations. This will cause confusion if, for example, an agency works on a taskforce with five other agencies. Would the agency count the taskforce as five organizations, or one partnership?

Module 3 – Community Level: (pages 28-50)

Module 3, Section A: Strategies Popup Windows: (page # 33-36)

p. 34 – Housing Strategies – I’d recommend including permanent affordable housing units to the second item on the list, which currently reads, “Create New Affordable Housing (SROs, temporary housing, transitional housing.”

Module 4 - Individual and Family Level: (pages 51-75)

Module 4, Section B: All Characteristics Report: (page # 53-54)

3. Education Levels: I do not believe it is useful to track the education level of individuals ages 14-24 as a group. Since this group includes both minors and young adults, data about the group’s education level would not be meaningful. For example, having an 8th grade education is entirely different for a 14-year-old who is currently in 9th grade and on track to graduate from high school than it is for a 22-year-old individual.

4. Disconnected Youth: This is data is a very good addition to the demographic form.

7. Military Status: This is data is a very good addition to the demographic form.

15. Non-Cash Benefits: I believe this is very useful information to track. I know that some agencies in my state’s network have expressed the view that collecting this information will be overly burdensome, but if nothing else it will cue agency staff to help people connect to benefits for which they are eligible. It

Comment 134 (York County Community Action ; Eligible Entity/Local Agency in ME)

The proposed collection of information is seen as overly burdensome, however, we are in agreement that a stronger focus on outcomes rather than outputs is a desirable goal. To meet that goal, we recommend the following:

The agency’s estimated time burden of 242 hours per sub-grantee (local Community Action Agency) is underestimated. Based on our calculations, the burden to our local agency would be 650 hours. The cost of this time burden is estimated to be $35,000.
While we greatly appreciate the idea of having an online automated system for use by states, the system will likely benefit the State Lead Agency and not the sub-grantee (Community

Action Agency). We will still need databases that track clients, services, outcomes, and many funders require separate data collection systems. Any new online system may in fact lead to added work hours.

20